{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b3d4d608",
   "metadata": {},
   "source": [
    "# Optuna一种超参数优化框架\n",
    "https://github.com/optuna/optuna\n",
    "\n",
    "分别采用两组数据集进行比较，分别是加利福尼亚住房数据集（回归）和森林植被类型（多分类）"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b22b6f90",
   "metadata": {},
   "source": [
    "## 结论：效果不错"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2d41e62",
   "metadata": {},
   "source": [
    "## 波士顿房价预测任务（回归）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "f5e0e977",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import time\n",
    "import gc\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import mean_squared_error\n",
    "import lightgbm as lgb  # 使用lgb模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "5eda2637",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import fetch_california_housing\n",
    "data = fetch_california_housing()\n",
    "X, y = data['data'], data['target']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "722beb8e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MedInc</th>\n",
       "      <th>HouseAge</th>\n",
       "      <th>AveRooms</th>\n",
       "      <th>AveBedrms</th>\n",
       "      <th>Population</th>\n",
       "      <th>AveOccup</th>\n",
       "      <th>Latitude</th>\n",
       "      <th>Longitude</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>8.3252</td>\n",
       "      <td>41.0</td>\n",
       "      <td>6.984127</td>\n",
       "      <td>1.02381</td>\n",
       "      <td>322.0</td>\n",
       "      <td>2.555556</td>\n",
       "      <td>37.88</td>\n",
       "      <td>-122.23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8.3014</td>\n",
       "      <td>21.0</td>\n",
       "      <td>6.238137</td>\n",
       "      <td>0.97188</td>\n",
       "      <td>2401.0</td>\n",
       "      <td>2.109842</td>\n",
       "      <td>37.86</td>\n",
       "      <td>-122.22</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   MedInc  HouseAge  AveRooms  AveBedrms  Population  AveOccup  Latitude  \\\n",
       "0  8.3252      41.0  6.984127    1.02381       322.0  2.555556     37.88   \n",
       "1  8.3014      21.0  6.238137    0.97188      2401.0  2.109842     37.86   \n",
       "\n",
       "   Longitude  \n",
       "0    -122.23  \n",
       "1    -122.22  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = pd.DataFrame(X,columns=data.feature_names)\n",
    "X.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "08ebab89",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 20640 entries, 0 to 20639\n",
      "Data columns (total 8 columns):\n",
      " #   Column      Non-Null Count  Dtype  \n",
      "---  ------      --------------  -----  \n",
      " 0   MedInc      20640 non-null  float64\n",
      " 1   HouseAge    20640 non-null  float64\n",
      " 2   AveRooms    20640 non-null  float64\n",
      " 3   AveBedrms   20640 non-null  float64\n",
      " 4   Population  20640 non-null  float64\n",
      " 5   AveOccup    20640 non-null  float64\n",
      " 6   Latitude    20640 non-null  float64\n",
      " 7   Longitude   20640 non-null  float64\n",
      "dtypes: float64(8)\n",
      "memory usage: 1.3 MB\n"
     ]
    }
   ],
   "source": [
    "X.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8932c66c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练集： (18576, 8)\n",
      "测试集： (2064, 8)\n"
     ]
    }
   ],
   "source": [
    "# 切分训练和测试集\n",
    "train_x, test_x, train_y, test_y = train_test_split(X, y,random_state=42,test_size=0.1)\n",
    "train_y = pd.DataFrame(train_y,columns=['result'])\n",
    "test_y = pd.DataFrame(test_y,columns=['result'])\n",
    "\n",
    "print('训练集：',train_x.shape)\n",
    "print('测试集：', test_x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "94561283",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import KFold\n",
    "skf = KFold(n_splits=5, shuffle=True, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e87a2121",
   "metadata": {},
   "source": [
    "### 使用LGB作为模型并CV，不使用optuna调参"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "5cad8967",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000866 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 1837\n",
      "[LightGBM] [Info] Number of data points in the train set: 14860, number of used features: 8\n",
      "[LightGBM] [Info] Start training from score 2.073247\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's rmse: 0.492877\n",
      "[100]\tvalid_0's rmse: 0.469235\n",
      "[150]\tvalid_0's rmse: 0.460357\n",
      "[200]\tvalid_0's rmse: 0.455302\n",
      "[250]\tvalid_0's rmse: 0.452674\n",
      "[300]\tvalid_0's rmse: 0.451179\n",
      "Early stopping, best iteration is:\n",
      "[313]\tvalid_0's rmse: 0.450303\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000494 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 1838\n",
      "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
      "[LightGBM] [Info] Start training from score 2.070396\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's rmse: 0.496484\n",
      "[100]\tvalid_0's rmse: 0.476225\n",
      "[150]\tvalid_0's rmse: 0.469582\n",
      "[200]\tvalid_0's rmse: 0.464643\n",
      "[250]\tvalid_0's rmse: 0.463244\n",
      "[300]\tvalid_0's rmse: 0.461886\n",
      "[350]\tvalid_0's rmse: 0.460305\n",
      "[400]\tvalid_0's rmse: 0.4596\n",
      "[450]\tvalid_0's rmse: 0.458645\n",
      "Early stopping, best iteration is:\n",
      "[435]\tvalid_0's rmse: 0.458368\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000800 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 1837\n",
      "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
      "[LightGBM] [Info] Start training from score 2.069154\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's rmse: 0.48676\n",
      "[100]\tvalid_0's rmse: 0.466911\n",
      "[150]\tvalid_0's rmse: 0.459693\n",
      "[200]\tvalid_0's rmse: 0.457631\n",
      "[250]\tvalid_0's rmse: 0.452574\n",
      "Early stopping, best iteration is:\n",
      "[264]\tvalid_0's rmse: 0.451947\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000779 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 1838\n",
      "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
      "[LightGBM] [Info] Start training from score 2.063366\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's rmse: 0.492482\n",
      "[100]\tvalid_0's rmse: 0.468929\n",
      "[150]\tvalid_0's rmse: 0.459713\n",
      "[200]\tvalid_0's rmse: 0.456203\n",
      "[250]\tvalid_0's rmse: 0.454063\n",
      "[300]\tvalid_0's rmse: 0.451901\n",
      "Early stopping, best iteration is:\n",
      "[322]\tvalid_0's rmse: 0.450961\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000694 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 1838\n",
      "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
      "[LightGBM] [Info] Start training from score 2.066924\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's rmse: 0.488385\n",
      "[100]\tvalid_0's rmse: 0.464851\n",
      "[150]\tvalid_0's rmse: 0.455183\n",
      "[200]\tvalid_0's rmse: 0.450506\n",
      "[250]\tvalid_0's rmse: 0.4461\n",
      "[300]\tvalid_0's rmse: 0.44438\n",
      "[350]\tvalid_0's rmse: 0.441808\n",
      "[400]\tvalid_0's rmse: 0.44079\n",
      "Early stopping, best iteration is:\n",
      "[411]\tvalid_0's rmse: 0.440543\n",
      "Wall time: 3.44 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "test_predict = np.zeros(shape=[test_x.shape[0], 5],dtype=float)\n",
    "params = {'boosting_type': 'gbdt',\n",
    "          'objective': 'regression',\n",
    "          \"metric\": 'rmse'}\n",
    "for i, (trn_idx, val_idx) in enumerate(skf.split(train_x, train_y)):\n",
    "    dtrain = lgb.Dataset(train_x.iloc[trn_idx], label=train_y.iloc[trn_idx])\n",
    "    dvalid = lgb.Dataset(train_x.iloc[val_idx], label=train_y.iloc[val_idx])\n",
    "    model = lgb.train(params=params, train_set=dtrain,valid_sets=[dvalid],\n",
    "                      verbose_eval=50,\n",
    "                      early_stopping_rounds=20,\n",
    "                      num_boost_round=5000)\n",
    "    test_predict[:,i] = model.predict(test_x)\n",
    "predict = np.mean(test_predict,axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "886bfdad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4346521330333544"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 评估指标rmse，越小越好\n",
    "np.sqrt(mean_squared_error(test_y,predict))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c271442c",
   "metadata": {},
   "source": [
    "### 使用LGB作为模型并CV，使用optuna调参"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "9a87f9db",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# pip install optuna"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "d8a0279a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import optuna"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "7433fdcb",
   "metadata": {},
   "outputs": [],
   "source": [
    "def objective(trial,train_x, train_y, valid_x, valid_y):\n",
    "    dtrain = lgb.Dataset(train_x, label=train_y)\n",
    "    dvalid = lgb.Dataset(valid_x, label=valid_y)\n",
    "\n",
    "    param = {\n",
    "        \"objective\": \"regression\",\n",
    "        \"metric\": \"rmse\",\n",
    "        \"verbosity\": -1,\n",
    "        \"boosting_type\": \"gbdt\",\n",
    "        'random_state':42,\n",
    "        \"lambda_l1\": trial.suggest_float(\"lambda_l1\", 1e-8, 10.0, log=True),\n",
    "        \"lambda_l2\": trial.suggest_float(\"lambda_l2\", 1e-8, 10.0, log=True),\n",
    "        \"num_leaves\": trial.suggest_int(\"num_leaves\", 2, 256),\n",
    "        \"feature_fraction\": trial.suggest_float(\"feature_fraction\", 0.4, 1.0),\n",
    "        \"bagging_fraction\": trial.suggest_float(\"bagging_fraction\", 0.4, 1.0),\n",
    "        \"bagging_freq\": trial.suggest_int(\"bagging_freq\", 1, 7),\n",
    "        \"min_child_samples\": trial.suggest_int(\"min_child_samples\", 5, 100),\n",
    "    }\n",
    "\n",
    "    # Add a callback for pruning.\n",
    "    pruning_callback = optuna.integration.LightGBMPruningCallback(trial, \"rmse\")\n",
    "    gbm = lgb.train(\n",
    "        param, dtrain, valid_sets=[dvalid], verbose_eval=False, callbacks=[pruning_callback]\n",
    "    )\n",
    "\n",
    "    preds = gbm.predict(valid_x)\n",
    "    pred_labels = np.rint(preds)\n",
    "    rmse = np.sqrt(mean_squared_error(valid_y,pred_labels))\n",
    "    return rmse"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "b9018adb",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 16:27:35,965]\u001b[0m A new study created in memory with name: no-name-cc424a48-83c6-4329-92a3-261d08a5edb9\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:36,779]\u001b[0m Trial 0 finished with value: 0.5178928385298298 and parameters: {'lambda_l1': 0.9464949700025425, 'lambda_l2': 1.4692321446797693e-05, 'num_leaves': 135, 'feature_fraction': 0.6448685329355106, 'bagging_fraction': 0.9048970068857253, 'bagging_freq': 7, 'min_child_samples': 30}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:37,641]\u001b[0m Trial 1 finished with value: 0.5336017055299863 and parameters: {'lambda_l1': 0.6354399847075458, 'lambda_l2': 4.87700419620481, 'num_leaves': 237, 'feature_fraction': 0.895991820221385, 'bagging_fraction': 0.6704364287317786, 'bagging_freq': 3, 'min_child_samples': 49}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:38,288]\u001b[0m Trial 2 finished with value: 0.5345721353318472 and parameters: {'lambda_l1': 9.52099217128906e-05, 'lambda_l2': 1.1454590646374039e-07, 'num_leaves': 170, 'feature_fraction': 0.7899599630691868, 'bagging_fraction': 0.5059741439893897, 'bagging_freq': 2, 'min_child_samples': 50}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:38,956]\u001b[0m Trial 3 finished with value: 0.543693048566443 and parameters: {'lambda_l1': 0.1481577233973877, 'lambda_l2': 3.179336503731134e-07, 'num_leaves': 128, 'feature_fraction': 0.9783519466402901, 'bagging_fraction': 0.6889127950806107, 'bagging_freq': 5, 'min_child_samples': 37}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:39,424]\u001b[0m Trial 4 finished with value: 0.5501760993688642 and parameters: {'lambda_l1': 3.7638689664291936, 'lambda_l2': 0.020384770050374573, 'num_leaves': 175, 'feature_fraction': 0.4840841073088404, 'bagging_fraction': 0.8334033110992205, 'bagging_freq': 5, 'min_child_samples': 35}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:39,493]\u001b[0m Trial 5 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:39,774]\u001b[0m Trial 6 pruned. Trial was pruned at iteration 33.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:39,828]\u001b[0m Trial 7 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:40,187]\u001b[0m Trial 8 finished with value: 0.5249490156742078 and parameters: {'lambda_l1': 8.186728837628555e-05, 'lambda_l2': 2.6479736415938615, 'num_leaves': 72, 'feature_fraction': 0.7599981859922114, 'bagging_fraction': 0.9692259725506919, 'bagging_freq': 5, 'min_child_samples': 38}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:40,299]\u001b[0m Trial 9 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:40,353]\u001b[0m Trial 10 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:40,398]\u001b[0m Trial 11 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:40,470]\u001b[0m Trial 12 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:41,280]\u001b[0m Trial 13 finished with value: 0.5325142394678348 and parameters: {'lambda_l1': 0.009551573559223583, 'lambda_l2': 0.0003114297759461472, 'num_leaves': 152, 'feature_fraction': 0.8236610445865283, 'bagging_fraction': 0.8389170404125451, 'bagging_freq': 4, 'min_child_samples': 20}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:41,391]\u001b[0m Trial 14 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:42,015]\u001b[0m Trial 15 finished with value: 0.5314302398651312 and parameters: {'lambda_l1': 0.024227381422510946, 'lambda_l2': 0.33276527010426776, 'num_leaves': 91, 'feature_fraction': 0.7874026441153832, 'bagging_fraction': 0.7899555875604796, 'bagging_freq': 7, 'min_child_samples': 19}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:42,091]\u001b[0m Trial 16 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:42,154]\u001b[0m Trial 17 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:42,226]\u001b[0m Trial 18 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:42,338]\u001b[0m Trial 19 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:42,384]\u001b[0m Trial 20 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:42,829]\u001b[0m Trial 21 finished with value: 0.5245233417218278 and parameters: {'lambda_l1': 0.07296901453015396, 'lambda_l2': 0.19272462963641587, 'num_leaves': 90, 'feature_fraction': 0.752955332537125, 'bagging_fraction': 0.7944607588474272, 'bagging_freq': 7, 'min_child_samples': 15}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:42,901]\u001b[0m Trial 22 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:43,287]\u001b[0m Trial 23 pruned. Trial was pruned at iteration 97.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:43,371]\u001b[0m Trial 24 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:43,489]\u001b[0m Trial 25 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:43,572]\u001b[0m Trial 26 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:43,632]\u001b[0m Trial 27 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:44,171]\u001b[0m Trial 28 finished with value: 0.5255422897694476 and parameters: {'lambda_l1': 0.028260092955639447, 'lambda_l2': 0.08413723627697263, 'num_leaves': 74, 'feature_fraction': 0.8042469147694201, 'bagging_fraction': 0.7548217217698655, 'bagging_freq': 3, 'min_child_samples': 42}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:44,331]\u001b[0m Trial 29 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:44,516]\u001b[0m Trial 30 pruned. Trial was pruned at iteration 16.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:44,637]\u001b[0m Trial 31 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:44,723]\u001b[0m Trial 32 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:45,592]\u001b[0m Trial 33 finished with value: 0.5218146276752049 and parameters: {'lambda_l1': 0.0016497886750146257, 'lambda_l2': 0.21484669274687151, 'num_leaves': 133, 'feature_fraction': 0.6885034648040802, 'bagging_fraction': 0.9281320338482011, 'bagging_freq': 3, 'min_child_samples': 42}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:45,711]\u001b[0m Trial 34 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:46,669]\u001b[0m Trial 35 finished with value: 0.5205542553680208 and parameters: {'lambda_l1': 4.3493248665689235e-05, 'lambda_l2': 3.680853885692289, 'num_leaves': 169, 'feature_fraction': 0.7104567429350184, 'bagging_fraction': 0.8672533271233644, 'bagging_freq': 1, 'min_child_samples': 23}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:47,676]\u001b[0m Trial 36 finished with value: 0.5206023202395521 and parameters: {'lambda_l1': 0.0006187306261428898, 'lambda_l2': 6.130428276943865e-08, 'num_leaves': 168, 'feature_fraction': 0.7036284508576037, 'bagging_fraction': 0.868428878180794, 'bagging_freq': 2, 'min_child_samples': 15}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:47,808]\u001b[0m Trial 37 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:47,937]\u001b[0m Trial 38 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:48,844]\u001b[0m Trial 39 finished with value: 0.5191042263340773 and parameters: {'lambda_l1': 0.0005562321404093257, 'lambda_l2': 2.232696278872472e-08, 'num_leaves': 161, 'feature_fraction': 0.710527336424428, 'bagging_fraction': 0.9309118257124477, 'bagging_freq': 1, 'min_child_samples': 24}. Best is trial 0 with value: 0.5178928385298298.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:49,889]\u001b[0m Trial 40 finished with value: 0.5168505162425636 and parameters: {'lambda_l1': 0.0004441190594738051, 'lambda_l2': 1.0707946566246148e-08, 'num_leaves': 190, 'feature_fraction': 0.7132737394337006, 'bagging_fraction': 0.8296082050311246, 'bagging_freq': 1, 'min_child_samples': 16}. Best is trial 40 with value: 0.5168505162425636.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:50,929]\u001b[0m Trial 41 finished with value: 0.5262832333602245 and parameters: {'lambda_l1': 0.0004892025280466791, 'lambda_l2': 1.89573604152611e-08, 'num_leaves': 189, 'feature_fraction': 0.7134600565845959, 'bagging_fraction': 0.8334328552244646, 'bagging_freq': 1, 'min_child_samples': 16}. Best is trial 40 with value: 0.5168505162425636.\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 16:27:51,023]\u001b[0m Trial 42 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:51,805]\u001b[0m Trial 43 finished with value: 0.516065766031832 and parameters: {'lambda_l1': 0.00021869661941011055, 'lambda_l2': 1.0674672013946999e-08, 'num_leaves': 201, 'feature_fraction': 0.7039965594752932, 'bagging_fraction': 0.8165783106186105, 'bagging_freq': 1, 'min_child_samples': 10}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:51,940]\u001b[0m Trial 44 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:52,044]\u001b[0m Trial 45 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:52,218]\u001b[0m Trial 46 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:53,067]\u001b[0m Trial 47 finished with value: 0.5256815758791472 and parameters: {'lambda_l1': 6.162602167779085e-05, 'lambda_l2': 1.6462819728079273e-07, 'num_leaves': 201, 'feature_fraction': 0.7163097853297453, 'bagging_fraction': 0.9420618119436382, 'bagging_freq': 1, 'min_child_samples': 24}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:53,178]\u001b[0m Trial 48 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:53,762]\u001b[0m Trial 49 finished with value: 0.5213791636070063 and parameters: {'lambda_l1': 2.774750232046531e-07, 'lambda_l2': 2.4404405074646332e-08, 'num_leaves': 154, 'feature_fraction': 0.6882189607163384, 'bagging_fraction': 0.9871043449165129, 'bagging_freq': 1, 'min_child_samples': 9}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:53,875]\u001b[0m Trial 50 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:54,587]\u001b[0m Trial 51 finished with value: 0.5174618905113623 and parameters: {'lambda_l1': 0.000381507052985562, 'lambda_l2': 4.901891994722806e-08, 'num_leaves': 166, 'feature_fraction': 0.7117978094083592, 'bagging_fraction': 0.8956613846537137, 'bagging_freq': 1, 'min_child_samples': 12}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:55,304]\u001b[0m Trial 52 finished with value: 0.5219780933938188 and parameters: {'lambda_l1': 0.0003055316771241415, 'lambda_l2': 1.0975399369852818e-08, 'num_leaves': 175, 'feature_fraction': 0.7709159871191618, 'bagging_fraction': 0.8946659725371712, 'bagging_freq': 1, 'min_child_samples': 12}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:56,481]\u001b[0m Trial 53 finished with value: 0.522019841081264 and parameters: {'lambda_l1': 3.0853077659712017e-06, 'lambda_l2': 3.167435996190143e-08, 'num_leaves': 199, 'feature_fraction': 0.700370780125537, 'bagging_fraction': 0.9659440325481246, 'bagging_freq': 1, 'min_child_samples': 5}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:57,422]\u001b[0m Trial 54 finished with value: 0.5210662969810745 and parameters: {'lambda_l1': 7.857450656938453e-05, 'lambda_l2': 4.5159973730477044e-07, 'num_leaves': 153, 'feature_fraction': 0.7263108852936352, 'bagging_fraction': 0.936630215722916, 'bagging_freq': 1, 'min_child_samples': 19}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:57,568]\u001b[0m Trial 55 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:57,678]\u001b[0m Trial 56 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:57,803]\u001b[0m Trial 57 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:57,889]\u001b[0m Trial 58 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:58,284]\u001b[0m Trial 59 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:58,510]\u001b[0m Trial 60 pruned. Trial was pruned at iteration 15.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:59,683]\u001b[0m Trial 61 finished with value: 0.5238780309760502 and parameters: {'lambda_l1': 0.0005622649168938934, 'lambda_l2': 7.041773458860283e-08, 'num_leaves': 165, 'feature_fraction': 0.6971774484774765, 'bagging_fraction': 0.8845735572162268, 'bagging_freq': 2, 'min_child_samples': 16}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:27:59,867]\u001b[0m Trial 62 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:01,236]\u001b[0m Trial 63 finished with value: 0.5172018048571797 and parameters: {'lambda_l1': 4.4774427854289624e-05, 'lambda_l2': 1.0431434685856767e-07, 'num_leaves': 175, 'feature_fraction': 0.7067550902029666, 'bagging_fraction': 0.7960327498051027, 'bagging_freq': 2, 'min_child_samples': 16}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:01,369]\u001b[0m Trial 64 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:01,513]\u001b[0m Trial 65 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:02,393]\u001b[0m Trial 66 pruned. Trial was pruned at iteration 76.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:02,529]\u001b[0m Trial 67 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:02,721]\u001b[0m Trial 68 pruned. Trial was pruned at iteration 12.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:03,060]\u001b[0m Trial 69 pruned. Trial was pruned at iteration 44.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:03,135]\u001b[0m Trial 70 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:03,486]\u001b[0m Trial 71 pruned. Trial was pruned at iteration 56.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:03,587]\u001b[0m Trial 72 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:04,107]\u001b[0m Trial 73 pruned. Trial was pruned at iteration 68.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:04,203]\u001b[0m Trial 74 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:04,307]\u001b[0m Trial 75 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:04,631]\u001b[0m Trial 76 pruned. Trial was pruned at iteration 39.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:04,725]\u001b[0m Trial 77 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:04,831]\u001b[0m Trial 78 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:04,920]\u001b[0m Trial 79 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:04,992]\u001b[0m Trial 80 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:05,099]\u001b[0m Trial 81 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:05,766]\u001b[0m Trial 82 finished with value: 0.5178938881576146 and parameters: {'lambda_l1': 1.2203483524616316e-08, 'lambda_l2': 5.507928324956204e-07, 'num_leaves': 185, 'feature_fraction': 0.7219415365227362, 'bagging_fraction': 0.9438920094595108, 'bagging_freq': 1, 'min_child_samples': 17}. Best is trial 43 with value: 0.516065766031832.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:05,866]\u001b[0m Trial 83 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:05,961]\u001b[0m Trial 84 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:06,778]\u001b[0m Trial 85 finished with value: 0.5122528701381253 and parameters: {'lambda_l1': 6.371822640459341e-07, 'lambda_l2': 3.03967794068636e-08, 'num_leaves': 189, 'feature_fraction': 0.7066521328434867, 'bagging_fraction': 0.8972190570020028, 'bagging_freq': 1, 'min_child_samples': 11}. Best is trial 85 with value: 0.5122528701381253.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:07,709]\u001b[0m Trial 86 finished with value: 0.5128398316765018 and parameters: {'lambda_l1': 1.5059254193489838e-07, 'lambda_l2': 2.6171717314455622e-08, 'num_leaves': 192, 'feature_fraction': 0.711268040370677, 'bagging_fraction': 0.9174432395706226, 'bagging_freq': 1, 'min_child_samples': 11}. Best is trial 85 with value: 0.5122528701381253.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:09,411]\u001b[0m Trial 87 finished with value: 0.5191456815789754 and parameters: {'lambda_l1': 5.687878085793806e-08, 'lambda_l2': 2.6772763929664963e-08, 'num_leaves': 216, 'feature_fraction': 0.7413398142797374, 'bagging_fraction': 0.9993988673163303, 'bagging_freq': 1, 'min_child_samples': 11}. Best is trial 85 with value: 0.5122528701381253.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:09,813]\u001b[0m Trial 88 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:12,526]\u001b[0m Trial 89 finished with value: 0.5102536883069437 and parameters: {'lambda_l1': 1.6446541932580473e-07, 'lambda_l2': 1.0734412329614426e-07, 'num_leaves': 193, 'feature_fraction': 0.8123395009389368, 'bagging_fraction': 0.9184649865458648, 'bagging_freq': 1, 'min_child_samples': 10}. Best is trial 89 with value: 0.5102536883069437.\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 16:28:12,792]\u001b[0m Trial 90 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:13,099]\u001b[0m Trial 91 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:13,456]\u001b[0m Trial 92 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:13,664]\u001b[0m Trial 93 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:14,351]\u001b[0m Trial 94 pruned. Trial was pruned at iteration 45.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:14,508]\u001b[0m Trial 95 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:14,828]\u001b[0m Trial 96 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:14,962]\u001b[0m Trial 97 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:16,197]\u001b[0m Trial 98 finished with value: 0.5171555001746302 and parameters: {'lambda_l1': 1.0103137663726183e-08, 'lambda_l2': 1.7418953701287595e-06, 'num_leaves': 203, 'feature_fraction': 0.7265189263478641, 'bagging_fraction': 0.9317931154547426, 'bagging_freq': 1, 'min_child_samples': 10}. Best is trial 89 with value: 0.5102536883069437.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:16,557]\u001b[0m Trial 99 pruned. Trial was pruned at iteration 16.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of finished trials: 100\n",
      "Best trial:\n",
      "  Value: 0.5102536883069437\n",
      "  Params: \n",
      "    lambda_l1: 1.6446541932580473e-07\n",
      "    lambda_l2: 1.0734412329614426e-07\n",
      "    num_leaves: 193\n",
      "    feature_fraction: 0.8123395009389368\n",
      "    bagging_fraction: 0.9184649865458648\n",
      "    bagging_freq: 1\n",
      "    min_child_samples: 10\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000712 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 1837\n",
      "[LightGBM] [Info] Number of data points in the train set: 14860, number of used features: 8\n",
      "[LightGBM] [Info] Start training from score 2.073247\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's rmse: 0.453164\n",
      "[100]\tvalid_0's rmse: 0.442958\n",
      "Early stopping, best iteration is:\n",
      "[110]\tvalid_0's rmse: 0.442157\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 16:28:17,677]\u001b[0m Trial 100 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:17,796]\u001b[0m Trial 101 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:17,941]\u001b[0m Trial 102 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:18,090]\u001b[0m Trial 103 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:18,293]\u001b[0m Trial 104 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:18,412]\u001b[0m Trial 105 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:18,524]\u001b[0m Trial 106 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:18,647]\u001b[0m Trial 107 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:18,797]\u001b[0m Trial 108 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:18,911]\u001b[0m Trial 109 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:19,029]\u001b[0m Trial 110 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:19,211]\u001b[0m Trial 111 pruned. Trial was pruned at iteration 16.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:19,595]\u001b[0m Trial 112 pruned. Trial was pruned at iteration 30.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:19,744]\u001b[0m Trial 113 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:19,853]\u001b[0m Trial 114 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:20,146]\u001b[0m Trial 115 pruned. Trial was pruned at iteration 23.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:20,267]\u001b[0m Trial 116 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:20,420]\u001b[0m Trial 117 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:20,594]\u001b[0m Trial 118 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:20,780]\u001b[0m Trial 119 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:20,942]\u001b[0m Trial 120 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:21,073]\u001b[0m Trial 121 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:21,410]\u001b[0m Trial 122 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:21,601]\u001b[0m Trial 123 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:21,799]\u001b[0m Trial 124 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:21,944]\u001b[0m Trial 125 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:22,080]\u001b[0m Trial 126 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:22,209]\u001b[0m Trial 127 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:22,367]\u001b[0m Trial 128 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:22,509]\u001b[0m Trial 129 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:22,673]\u001b[0m Trial 130 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:22,754]\u001b[0m Trial 131 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:22,923]\u001b[0m Trial 132 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:23,074]\u001b[0m Trial 133 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:23,442]\u001b[0m Trial 134 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:23,682]\u001b[0m Trial 135 pruned. Trial was pruned at iteration 16.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:23,824]\u001b[0m Trial 136 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:24,010]\u001b[0m Trial 137 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:24,160]\u001b[0m Trial 138 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:24,458]\u001b[0m Trial 139 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:24,678]\u001b[0m Trial 140 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:24,849]\u001b[0m Trial 141 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:25,081]\u001b[0m Trial 142 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:25,256]\u001b[0m Trial 143 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:25,501]\u001b[0m Trial 144 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:26,208]\u001b[0m Trial 145 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:26,709]\u001b[0m Trial 146 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:26,919]\u001b[0m Trial 147 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:27,534]\u001b[0m Trial 148 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:27,729]\u001b[0m Trial 149 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:28,152]\u001b[0m Trial 150 pruned. Trial was pruned at iteration 24.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:28,278]\u001b[0m Trial 151 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:28,393]\u001b[0m Trial 152 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:28,543]\u001b[0m Trial 153 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:28,653]\u001b[0m Trial 154 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:28,757]\u001b[0m Trial 155 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:28,887]\u001b[0m Trial 156 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:29,001]\u001b[0m Trial 157 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:29,176]\u001b[0m Trial 158 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:29,332]\u001b[0m Trial 159 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:29,541]\u001b[0m Trial 160 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:29,732]\u001b[0m Trial 161 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:29,884]\u001b[0m Trial 162 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:29,997]\u001b[0m Trial 163 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:30,103]\u001b[0m Trial 164 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:30,225]\u001b[0m Trial 165 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:30,330]\u001b[0m Trial 166 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:30,487]\u001b[0m Trial 167 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:30,579]\u001b[0m Trial 168 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:30,685]\u001b[0m Trial 169 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:30,798]\u001b[0m Trial 170 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:30,902]\u001b[0m Trial 171 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:30,999]\u001b[0m Trial 172 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:31,099]\u001b[0m Trial 173 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:31,206]\u001b[0m Trial 174 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:31,352]\u001b[0m Trial 175 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:31,476]\u001b[0m Trial 176 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:31,716]\u001b[0m Trial 177 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:31,834]\u001b[0m Trial 178 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:31,983]\u001b[0m Trial 179 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:32,099]\u001b[0m Trial 180 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:32,272]\u001b[0m Trial 181 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:32,431]\u001b[0m Trial 182 pruned. Trial was pruned at iteration 12.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:32,546]\u001b[0m Trial 183 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:32,656]\u001b[0m Trial 184 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:32,763]\u001b[0m Trial 185 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:32,965]\u001b[0m Trial 186 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:33,183]\u001b[0m Trial 187 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:33,409]\u001b[0m Trial 188 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 16:28:33,571]\u001b[0m Trial 189 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:33,719]\u001b[0m Trial 190 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:33,919]\u001b[0m Trial 191 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:34,112]\u001b[0m Trial 192 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:34,276]\u001b[0m Trial 193 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:34,426]\u001b[0m Trial 194 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:34,633]\u001b[0m Trial 195 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:34,781]\u001b[0m Trial 196 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:35,002]\u001b[0m Trial 197 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:35,242]\u001b[0m Trial 198 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:35,408]\u001b[0m Trial 199 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of finished trials: 200\n",
      "Best trial:\n",
      "  Value: 0.5102536883069437\n",
      "  Params: \n",
      "    lambda_l1: 1.6446541932580473e-07\n",
      "    lambda_l2: 1.0734412329614426e-07\n",
      "    num_leaves: 193\n",
      "    feature_fraction: 0.8123395009389368\n",
      "    bagging_fraction: 0.9184649865458648\n",
      "    bagging_freq: 1\n",
      "    min_child_samples: 10\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000777 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 1838\n",
      "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
      "[LightGBM] [Info] Start training from score 2.070396\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's rmse: 0.465875\n",
      "Early stopping, best iteration is:\n",
      "[76]\tvalid_0's rmse: 0.460117\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 16:28:36,810]\u001b[0m Trial 200 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:36,973]\u001b[0m Trial 201 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:37,136]\u001b[0m Trial 202 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:37,255]\u001b[0m Trial 203 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:37,402]\u001b[0m Trial 204 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:37,545]\u001b[0m Trial 205 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:37,742]\u001b[0m Trial 206 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:37,864]\u001b[0m Trial 207 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:38,005]\u001b[0m Trial 208 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:38,155]\u001b[0m Trial 209 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:38,307]\u001b[0m Trial 210 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:38,419]\u001b[0m Trial 211 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:38,537]\u001b[0m Trial 212 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:38,637]\u001b[0m Trial 213 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:38,786]\u001b[0m Trial 214 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:38,962]\u001b[0m Trial 215 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:39,091]\u001b[0m Trial 216 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:39,246]\u001b[0m Trial 217 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:39,368]\u001b[0m Trial 218 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:39,505]\u001b[0m Trial 219 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:39,637]\u001b[0m Trial 220 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:39,737]\u001b[0m Trial 221 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:39,847]\u001b[0m Trial 222 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:39,948]\u001b[0m Trial 223 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:40,046]\u001b[0m Trial 224 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:40,138]\u001b[0m Trial 225 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:40,235]\u001b[0m Trial 226 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:40,324]\u001b[0m Trial 227 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:40,398]\u001b[0m Trial 228 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:40,483]\u001b[0m Trial 229 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:40,621]\u001b[0m Trial 230 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:40,779]\u001b[0m Trial 231 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:40,860]\u001b[0m Trial 232 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:40,937]\u001b[0m Trial 233 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:41,077]\u001b[0m Trial 234 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:41,188]\u001b[0m Trial 235 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:41,312]\u001b[0m Trial 236 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:41,431]\u001b[0m Trial 237 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:41,533]\u001b[0m Trial 238 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:41,664]\u001b[0m Trial 239 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:41,790]\u001b[0m Trial 240 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:41,902]\u001b[0m Trial 241 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:42,009]\u001b[0m Trial 242 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:42,110]\u001b[0m Trial 243 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:42,233]\u001b[0m Trial 244 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:42,443]\u001b[0m Trial 245 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:42,587]\u001b[0m Trial 246 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:42,674]\u001b[0m Trial 247 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:42,779]\u001b[0m Trial 248 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:42,859]\u001b[0m Trial 249 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:42,958]\u001b[0m Trial 250 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:43,070]\u001b[0m Trial 251 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:43,257]\u001b[0m Trial 252 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:43,356]\u001b[0m Trial 253 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:43,462]\u001b[0m Trial 254 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:43,566]\u001b[0m Trial 255 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:43,627]\u001b[0m Trial 256 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:43,743]\u001b[0m Trial 257 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:43,856]\u001b[0m Trial 258 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:43,945]\u001b[0m Trial 259 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:44,059]\u001b[0m Trial 260 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:44,164]\u001b[0m Trial 261 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:44,279]\u001b[0m Trial 262 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:44,377]\u001b[0m Trial 263 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:44,478]\u001b[0m Trial 264 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:44,585]\u001b[0m Trial 265 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:44,688]\u001b[0m Trial 266 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:44,794]\u001b[0m Trial 267 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:44,914]\u001b[0m Trial 268 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:45,016]\u001b[0m Trial 269 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:45,111]\u001b[0m Trial 270 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:45,248]\u001b[0m Trial 271 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:45,424]\u001b[0m Trial 272 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:45,578]\u001b[0m Trial 273 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:45,809]\u001b[0m Trial 274 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:45,937]\u001b[0m Trial 275 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:46,068]\u001b[0m Trial 276 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:46,210]\u001b[0m Trial 277 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:46,357]\u001b[0m Trial 278 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:46,505]\u001b[0m Trial 279 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:46,662]\u001b[0m Trial 280 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:46,828]\u001b[0m Trial 281 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:46,927]\u001b[0m Trial 282 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:47,157]\u001b[0m Trial 283 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:47,394]\u001b[0m Trial 284 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:47,564]\u001b[0m Trial 285 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:47,701]\u001b[0m Trial 286 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:47,936]\u001b[0m Trial 287 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:48,091]\u001b[0m Trial 288 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 16:28:48,268]\u001b[0m Trial 289 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:48,470]\u001b[0m Trial 290 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:48,650]\u001b[0m Trial 291 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:48,799]\u001b[0m Trial 292 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:48,953]\u001b[0m Trial 293 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:49,112]\u001b[0m Trial 294 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:49,267]\u001b[0m Trial 295 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:49,454]\u001b[0m Trial 296 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:49,631]\u001b[0m Trial 297 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:49,744]\u001b[0m Trial 298 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:49,933]\u001b[0m Trial 299 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of finished trials: 300\n",
      "Best trial:\n",
      "  Value: 0.5102536883069437\n",
      "  Params: \n",
      "    lambda_l1: 1.6446541932580473e-07\n",
      "    lambda_l2: 1.0734412329614426e-07\n",
      "    num_leaves: 193\n",
      "    feature_fraction: 0.8123395009389368\n",
      "    bagging_fraction: 0.9184649865458648\n",
      "    bagging_freq: 1\n",
      "    min_child_samples: 10\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000797 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 1837\n",
      "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
      "[LightGBM] [Info] Start training from score 2.069154\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's rmse: 0.451573\n",
      "Early stopping, best iteration is:\n",
      "[63]\tvalid_0's rmse: 0.44707\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 16:28:51,004]\u001b[0m Trial 300 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:51,159]\u001b[0m Trial 301 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:51,315]\u001b[0m Trial 302 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:51,412]\u001b[0m Trial 303 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:51,543]\u001b[0m Trial 304 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:51,686]\u001b[0m Trial 305 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:51,842]\u001b[0m Trial 306 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:51,977]\u001b[0m Trial 307 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:52,105]\u001b[0m Trial 308 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:52,202]\u001b[0m Trial 309 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:52,341]\u001b[0m Trial 310 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:52,426]\u001b[0m Trial 311 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:52,522]\u001b[0m Trial 312 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:52,642]\u001b[0m Trial 313 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:52,755]\u001b[0m Trial 314 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:52,880]\u001b[0m Trial 315 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:52,980]\u001b[0m Trial 316 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:53,093]\u001b[0m Trial 317 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:53,202]\u001b[0m Trial 318 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:53,333]\u001b[0m Trial 319 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:53,488]\u001b[0m Trial 320 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:53,634]\u001b[0m Trial 321 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:53,749]\u001b[0m Trial 322 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:53,863]\u001b[0m Trial 323 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:53,983]\u001b[0m Trial 324 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:54,088]\u001b[0m Trial 325 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:54,198]\u001b[0m Trial 326 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:54,321]\u001b[0m Trial 327 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:54,417]\u001b[0m Trial 328 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:54,486]\u001b[0m Trial 329 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:54,630]\u001b[0m Trial 330 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:54,711]\u001b[0m Trial 331 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:54,838]\u001b[0m Trial 332 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:54,968]\u001b[0m Trial 333 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:55,077]\u001b[0m Trial 334 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:55,179]\u001b[0m Trial 335 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:55,319]\u001b[0m Trial 336 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:55,518]\u001b[0m Trial 337 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:55,609]\u001b[0m Trial 338 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:55,741]\u001b[0m Trial 339 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:55,903]\u001b[0m Trial 340 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:56,024]\u001b[0m Trial 341 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:56,140]\u001b[0m Trial 342 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:56,425]\u001b[0m Trial 343 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:56,577]\u001b[0m Trial 344 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:56,697]\u001b[0m Trial 345 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:56,814]\u001b[0m Trial 346 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:56,931]\u001b[0m Trial 347 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:57,047]\u001b[0m Trial 348 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:57,165]\u001b[0m Trial 349 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:57,318]\u001b[0m Trial 350 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:57,470]\u001b[0m Trial 351 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:57,640]\u001b[0m Trial 352 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:57,826]\u001b[0m Trial 353 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:57,991]\u001b[0m Trial 354 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:58,135]\u001b[0m Trial 355 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:58,300]\u001b[0m Trial 356 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:58,479]\u001b[0m Trial 357 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:58,638]\u001b[0m Trial 358 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:58,813]\u001b[0m Trial 359 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:59,027]\u001b[0m Trial 360 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:59,183]\u001b[0m Trial 361 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:59,346]\u001b[0m Trial 362 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:59,510]\u001b[0m Trial 363 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:59,627]\u001b[0m Trial 364 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:59,792]\u001b[0m Trial 365 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:28:59,954]\u001b[0m Trial 366 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:00,188]\u001b[0m Trial 367 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:00,393]\u001b[0m Trial 368 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:00,557]\u001b[0m Trial 369 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:00,707]\u001b[0m Trial 370 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:00,836]\u001b[0m Trial 371 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:01,018]\u001b[0m Trial 372 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:01,171]\u001b[0m Trial 373 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:01,355]\u001b[0m Trial 374 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:01,634]\u001b[0m Trial 375 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:01,803]\u001b[0m Trial 376 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:01,949]\u001b[0m Trial 377 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:02,110]\u001b[0m Trial 378 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:02,225]\u001b[0m Trial 379 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:02,409]\u001b[0m Trial 380 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:02,574]\u001b[0m Trial 381 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:02,733]\u001b[0m Trial 382 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:02,895]\u001b[0m Trial 383 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:03,059]\u001b[0m Trial 384 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:03,184]\u001b[0m Trial 385 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:03,408]\u001b[0m Trial 386 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:03,583]\u001b[0m Trial 387 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:03,729]\u001b[0m Trial 388 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 16:29:03,924]\u001b[0m Trial 389 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:04,120]\u001b[0m Trial 390 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:04,292]\u001b[0m Trial 391 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:04,406]\u001b[0m Trial 392 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:04,514]\u001b[0m Trial 393 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:04,671]\u001b[0m Trial 394 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:04,760]\u001b[0m Trial 395 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:04,879]\u001b[0m Trial 396 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:05,049]\u001b[0m Trial 397 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:05,188]\u001b[0m Trial 398 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:05,277]\u001b[0m Trial 399 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of finished trials: 400\n",
      "Best trial:\n",
      "  Value: 0.5102536883069437\n",
      "  Params: \n",
      "    lambda_l1: 1.6446541932580473e-07\n",
      "    lambda_l2: 1.0734412329614426e-07\n",
      "    num_leaves: 193\n",
      "    feature_fraction: 0.8123395009389368\n",
      "    bagging_fraction: 0.9184649865458648\n",
      "    bagging_freq: 1\n",
      "    min_child_samples: 10\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000424 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 1838\n",
      "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
      "[LightGBM] [Info] Start training from score 2.063366\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's rmse: 0.458105\n",
      "[100]\tvalid_0's rmse: 0.449598\n",
      "[150]\tvalid_0's rmse: 0.449092\n",
      "Early stopping, best iteration is:\n",
      "[138]\tvalid_0's rmse: 0.448682\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 16:29:06,435]\u001b[0m Trial 400 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:06,586]\u001b[0m Trial 401 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:06,756]\u001b[0m Trial 402 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:06,867]\u001b[0m Trial 403 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:06,968]\u001b[0m Trial 404 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:07,051]\u001b[0m Trial 405 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:07,151]\u001b[0m Trial 406 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:07,290]\u001b[0m Trial 407 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:07,400]\u001b[0m Trial 408 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:07,530]\u001b[0m Trial 409 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:07,649]\u001b[0m Trial 410 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:07,752]\u001b[0m Trial 411 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:07,880]\u001b[0m Trial 412 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:08,002]\u001b[0m Trial 413 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:08,140]\u001b[0m Trial 414 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:08,264]\u001b[0m Trial 415 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:08,434]\u001b[0m Trial 416 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:08,553]\u001b[0m Trial 417 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:08,677]\u001b[0m Trial 418 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:08,794]\u001b[0m Trial 419 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:08,902]\u001b[0m Trial 420 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:09,060]\u001b[0m Trial 421 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:09,181]\u001b[0m Trial 422 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:09,279]\u001b[0m Trial 423 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:09,517]\u001b[0m Trial 424 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:09,691]\u001b[0m Trial 425 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:09,858]\u001b[0m Trial 426 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:10,005]\u001b[0m Trial 427 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:10,197]\u001b[0m Trial 428 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:10,377]\u001b[0m Trial 429 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:10,567]\u001b[0m Trial 430 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:10,749]\u001b[0m Trial 431 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:10,908]\u001b[0m Trial 432 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:11,012]\u001b[0m Trial 433 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:11,264]\u001b[0m Trial 434 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:11,460]\u001b[0m Trial 435 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:11,636]\u001b[0m Trial 436 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:11,805]\u001b[0m Trial 437 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:11,985]\u001b[0m Trial 438 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:12,116]\u001b[0m Trial 439 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:12,257]\u001b[0m Trial 440 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:12,455]\u001b[0m Trial 441 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:12,626]\u001b[0m Trial 442 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:12,785]\u001b[0m Trial 443 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:12,916]\u001b[0m Trial 444 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:13,059]\u001b[0m Trial 445 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:13,202]\u001b[0m Trial 446 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:13,402]\u001b[0m Trial 447 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:13,571]\u001b[0m Trial 448 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:13,751]\u001b[0m Trial 449 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:13,967]\u001b[0m Trial 450 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:14,171]\u001b[0m Trial 451 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:14,354]\u001b[0m Trial 452 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:14,580]\u001b[0m Trial 453 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:14,763]\u001b[0m Trial 454 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:14,944]\u001b[0m Trial 455 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:15,124]\u001b[0m Trial 456 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:15,326]\u001b[0m Trial 457 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:15,508]\u001b[0m Trial 458 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:15,707]\u001b[0m Trial 459 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:15,875]\u001b[0m Trial 460 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:16,043]\u001b[0m Trial 461 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:16,205]\u001b[0m Trial 462 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:16,355]\u001b[0m Trial 463 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:16,499]\u001b[0m Trial 464 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:16,601]\u001b[0m Trial 465 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:16,757]\u001b[0m Trial 466 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:16,899]\u001b[0m Trial 467 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:17,040]\u001b[0m Trial 468 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:17,256]\u001b[0m Trial 469 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:17,440]\u001b[0m Trial 470 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:17,613]\u001b[0m Trial 471 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:17,735]\u001b[0m Trial 472 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:17,851]\u001b[0m Trial 473 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:17,940]\u001b[0m Trial 474 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:18,015]\u001b[0m Trial 475 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:18,140]\u001b[0m Trial 476 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:18,318]\u001b[0m Trial 477 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:18,485]\u001b[0m Trial 478 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:18,594]\u001b[0m Trial 479 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:18,767]\u001b[0m Trial 480 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:18,864]\u001b[0m Trial 481 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:19,000]\u001b[0m Trial 482 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:19,141]\u001b[0m Trial 483 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:19,241]\u001b[0m Trial 484 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:19,362]\u001b[0m Trial 485 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:19,512]\u001b[0m Trial 486 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:19,651]\u001b[0m Trial 487 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:19,777]\u001b[0m Trial 488 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 16:29:19,892]\u001b[0m Trial 489 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:20,026]\u001b[0m Trial 490 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:20,152]\u001b[0m Trial 491 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:20,291]\u001b[0m Trial 492 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:20,385]\u001b[0m Trial 493 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:20,508]\u001b[0m Trial 494 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:20,644]\u001b[0m Trial 495 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:20,766]\u001b[0m Trial 496 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:20,912]\u001b[0m Trial 497 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:21,093]\u001b[0m Trial 498 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 16:29:21,228]\u001b[0m Trial 499 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of finished trials: 500\n",
      "Best trial:\n",
      "  Value: 0.5102536883069437\n",
      "  Params: \n",
      "    lambda_l1: 1.6446541932580473e-07\n",
      "    lambda_l2: 1.0734412329614426e-07\n",
      "    num_leaves: 193\n",
      "    feature_fraction: 0.8123395009389368\n",
      "    bagging_fraction: 0.9184649865458648\n",
      "    bagging_freq: 1\n",
      "    min_child_samples: 10\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.001110 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 1838\n",
      "[LightGBM] [Info] Number of data points in the train set: 14861, number of used features: 8\n",
      "[LightGBM] [Info] Start training from score 2.066924\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's rmse: 0.444941\n",
      "[100]\tvalid_0's rmse: 0.437147\n",
      "Early stopping, best iteration is:\n",
      "[90]\tvalid_0's rmse: 0.43678\n",
      "Wall time: 1min 46s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "if __name__ == \"__main__\":\n",
    "    test_predict = np.zeros(shape=[test_x.shape[0], 5],dtype=float)\n",
    "    study = optuna.create_study(\n",
    "        pruner=optuna.pruners.MedianPruner(n_warmup_steps=10), direction=\"minimize\"  # 指定是越小越好\n",
    "    )\n",
    "    for i, (trn_idx, val_idx) in enumerate(skf.split(train_x, train_y)):\n",
    "        study.optimize(lambda trial: objective(trial, train_x.iloc[trn_idx], train_y.iloc[trn_idx], \n",
    "                                               train_x.iloc[val_idx], train_y.iloc[val_idx]), n_trials=100)\n",
    "\n",
    "        print(\"Number of finished trials: {}\".format(len(study.trials)))\n",
    "\n",
    "        print(\"Best trial:\")\n",
    "        trial = study.best_trial\n",
    "\n",
    "        print(\"  Value: {}\".format(trial.value))\n",
    "\n",
    "        print(\"  Params: \")\n",
    "        for key, value in trial.params.items():\n",
    "            print(\"    {}: {}\".format(key, value))\n",
    "            \n",
    "        params = {'boosting_type': 'gbdt',\n",
    "                  'objective': 'regression',\n",
    "                  \"metric\": 'rmse'}\n",
    "        for key, value in trial.params.items():\n",
    "            params[key]=value\n",
    "            \n",
    "        dtrain = lgb.Dataset(train_x.iloc[trn_idx], label=train_y.iloc[trn_idx])\n",
    "        dvalid = lgb.Dataset(train_x.iloc[val_idx], label=train_y.iloc[val_idx])\n",
    "        model = lgb.train(params=params, train_set=dtrain,valid_sets=[dvalid],\n",
    "                          verbose_eval=50,\n",
    "                          early_stopping_rounds=20,\n",
    "                          num_boost_round=5000)\n",
    "        test_predict[:,i] = model.predict(test_x)\n",
    "    predict = np.mean(test_predict,axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "f28d82da",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.432341765333029"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 评估指标rmse，越小越好\n",
    "np.sqrt(mean_squared_error(test_y,predict))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c7ee3c5",
   "metadata": {},
   "source": [
    "### 回归任务的结论\n",
    "不使用optuna的RMSE分数是0.4346521330333544，使用的RMSE分数是0.432341765333029，提升了0.0023103677003254。\n",
    "\n",
    "作者测了很多次，基本在0.003-0.002之间，感兴趣的可以多跑几次。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb67b354",
   "metadata": {},
   "source": [
    "## 森林植被类型（多分类）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "108949a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import time\n",
    "import gc\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.preprocessing import OrdinalEncoder\n",
    "import lightgbm as lgb  # 使用lgb模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "36e26941",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "七分类任务，处理前： [1 2 3 4 5 6 7]\n",
      "[5 5 2 ... 3 3 3]\n",
      "七分类任务，处理后： [0. 1. 2. 3. 4. 5. 6.]\n",
      "[4. 4. 1. ... 2. 2. 2.]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Elevation</th>\n",
       "      <th>Aspect</th>\n",
       "      <th>Slope</th>\n",
       "      <th>Horizontal_Distance_To_Hydrology</th>\n",
       "      <th>Vertical_Distance_To_Hydrology</th>\n",
       "      <th>Horizontal_Distance_To_Roadways</th>\n",
       "      <th>Hillshade_9am</th>\n",
       "      <th>Hillshade_Noon</th>\n",
       "      <th>Hillshade_3pm</th>\n",
       "      <th>Horizontal_Distance_To_Fire_Points</th>\n",
       "      <th>...</th>\n",
       "      <th>Soil_Type_30</th>\n",
       "      <th>Soil_Type_31</th>\n",
       "      <th>Soil_Type_32</th>\n",
       "      <th>Soil_Type_33</th>\n",
       "      <th>Soil_Type_34</th>\n",
       "      <th>Soil_Type_35</th>\n",
       "      <th>Soil_Type_36</th>\n",
       "      <th>Soil_Type_37</th>\n",
       "      <th>Soil_Type_38</th>\n",
       "      <th>Soil_Type_39</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2596.0</td>\n",
       "      <td>51.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>258.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>510.0</td>\n",
       "      <td>221.0</td>\n",
       "      <td>232.0</td>\n",
       "      <td>148.0</td>\n",
       "      <td>6279.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2590.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>212.0</td>\n",
       "      <td>-6.0</td>\n",
       "      <td>390.0</td>\n",
       "      <td>220.0</td>\n",
       "      <td>235.0</td>\n",
       "      <td>151.0</td>\n",
       "      <td>6225.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2 rows × 54 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Elevation  Aspect  Slope  Horizontal_Distance_To_Hydrology  \\\n",
       "0     2596.0    51.0    3.0                             258.0   \n",
       "1     2590.0    56.0    2.0                             212.0   \n",
       "\n",
       "   Vertical_Distance_To_Hydrology  Horizontal_Distance_To_Roadways  \\\n",
       "0                             0.0                            510.0   \n",
       "1                            -6.0                            390.0   \n",
       "\n",
       "   Hillshade_9am  Hillshade_Noon  Hillshade_3pm  \\\n",
       "0          221.0           232.0          148.0   \n",
       "1          220.0           235.0          151.0   \n",
       "\n",
       "   Horizontal_Distance_To_Fire_Points  ...  Soil_Type_30  Soil_Type_31  \\\n",
       "0                              6279.0  ...           0.0           0.0   \n",
       "1                              6225.0  ...           0.0           0.0   \n",
       "\n",
       "   Soil_Type_32  Soil_Type_33  Soil_Type_34  Soil_Type_35  Soil_Type_36  \\\n",
       "0           0.0           0.0           0.0           0.0           0.0   \n",
       "1           0.0           0.0           0.0           0.0           0.0   \n",
       "\n",
       "   Soil_Type_37  Soil_Type_38  Soil_Type_39  \n",
       "0           0.0           0.0           0.0  \n",
       "1           0.0           0.0           0.0  \n",
       "\n",
       "[2 rows x 54 columns]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.datasets import fetch_covtype\n",
    "data = fetch_covtype()\n",
    "X, y = data['data'], data['target']\n",
    "X = pd.DataFrame(X,columns=data.feature_names)\n",
    "print('七分类任务，处理前：',np.unique(y))\n",
    "print(y)\n",
    "ord = OrdinalEncoder()\n",
    "y = ord.fit_transform(y.reshape(-1, 1))\n",
    "y = y.reshape(-1, )\n",
    "print('七分类任务，处理后：',np.unique(y))\n",
    "print(y)\n",
    "X.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "152a8658",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 581012 entries, 0 to 581011\n",
      "Data columns (total 54 columns):\n",
      " #   Column                              Non-Null Count   Dtype  \n",
      "---  ------                              --------------   -----  \n",
      " 0   Elevation                           581012 non-null  float64\n",
      " 1   Aspect                              581012 non-null  float64\n",
      " 2   Slope                               581012 non-null  float64\n",
      " 3   Horizontal_Distance_To_Hydrology    581012 non-null  float64\n",
      " 4   Vertical_Distance_To_Hydrology      581012 non-null  float64\n",
      " 5   Horizontal_Distance_To_Roadways     581012 non-null  float64\n",
      " 6   Hillshade_9am                       581012 non-null  float64\n",
      " 7   Hillshade_Noon                      581012 non-null  float64\n",
      " 8   Hillshade_3pm                       581012 non-null  float64\n",
      " 9   Horizontal_Distance_To_Fire_Points  581012 non-null  float64\n",
      " 10  Wilderness_Area_0                   581012 non-null  float64\n",
      " 11  Wilderness_Area_1                   581012 non-null  float64\n",
      " 12  Wilderness_Area_2                   581012 non-null  float64\n",
      " 13  Wilderness_Area_3                   581012 non-null  float64\n",
      " 14  Soil_Type_0                         581012 non-null  float64\n",
      " 15  Soil_Type_1                         581012 non-null  float64\n",
      " 16  Soil_Type_2                         581012 non-null  float64\n",
      " 17  Soil_Type_3                         581012 non-null  float64\n",
      " 18  Soil_Type_4                         581012 non-null  float64\n",
      " 19  Soil_Type_5                         581012 non-null  float64\n",
      " 20  Soil_Type_6                         581012 non-null  float64\n",
      " 21  Soil_Type_7                         581012 non-null  float64\n",
      " 22  Soil_Type_8                         581012 non-null  float64\n",
      " 23  Soil_Type_9                         581012 non-null  float64\n",
      " 24  Soil_Type_10                        581012 non-null  float64\n",
      " 25  Soil_Type_11                        581012 non-null  float64\n",
      " 26  Soil_Type_12                        581012 non-null  float64\n",
      " 27  Soil_Type_13                        581012 non-null  float64\n",
      " 28  Soil_Type_14                        581012 non-null  float64\n",
      " 29  Soil_Type_15                        581012 non-null  float64\n",
      " 30  Soil_Type_16                        581012 non-null  float64\n",
      " 31  Soil_Type_17                        581012 non-null  float64\n",
      " 32  Soil_Type_18                        581012 non-null  float64\n",
      " 33  Soil_Type_19                        581012 non-null  float64\n",
      " 34  Soil_Type_20                        581012 non-null  float64\n",
      " 35  Soil_Type_21                        581012 non-null  float64\n",
      " 36  Soil_Type_22                        581012 non-null  float64\n",
      " 37  Soil_Type_23                        581012 non-null  float64\n",
      " 38  Soil_Type_24                        581012 non-null  float64\n",
      " 39  Soil_Type_25                        581012 non-null  float64\n",
      " 40  Soil_Type_26                        581012 non-null  float64\n",
      " 41  Soil_Type_27                        581012 non-null  float64\n",
      " 42  Soil_Type_28                        581012 non-null  float64\n",
      " 43  Soil_Type_29                        581012 non-null  float64\n",
      " 44  Soil_Type_30                        581012 non-null  float64\n",
      " 45  Soil_Type_31                        581012 non-null  float64\n",
      " 46  Soil_Type_32                        581012 non-null  float64\n",
      " 47  Soil_Type_33                        581012 non-null  float64\n",
      " 48  Soil_Type_34                        581012 non-null  float64\n",
      " 49  Soil_Type_35                        581012 non-null  float64\n",
      " 50  Soil_Type_36                        581012 non-null  float64\n",
      " 51  Soil_Type_37                        581012 non-null  float64\n",
      " 52  Soil_Type_38                        581012 non-null  float64\n",
      " 53  Soil_Type_39                        581012 non-null  float64\n",
      "dtypes: float64(54)\n",
      "memory usage: 239.4 MB\n"
     ]
    }
   ],
   "source": [
    "X.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "67086c1c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练集： (522910, 54)\n",
      "测试集： (58102, 54)\n"
     ]
    }
   ],
   "source": [
    "# 切分训练和测试集\n",
    "train_x, test_x, train_y, test_y = train_test_split(X, y,random_state=42,test_size=0.1)\n",
    "train_y = pd.DataFrame(train_y,columns=['result'])\n",
    "test_y = pd.DataFrame(test_y,columns=['result'])\n",
    "\n",
    "print('训练集：',train_x.shape)\n",
    "print('测试集：', test_x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "88350615",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import StratifiedKFold\n",
    "skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "2e694b0c",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.030610 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 2264\n",
      "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n",
      "[LightGBM] [Info] Start training from score -1.009534\n",
      "[LightGBM] [Info] Start training from score -0.717940\n",
      "[LightGBM] [Info] Start training from score -2.789075\n",
      "[LightGBM] [Info] Start training from score -5.347629\n",
      "[LightGBM] [Info] Start training from score -4.112163\n",
      "[LightGBM] [Info] Start training from score -3.508300\n",
      "[LightGBM] [Info] Start training from score -3.343939\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's multi_logloss: 0.424906\n",
      "[100]\tvalid_0's multi_logloss: 0.413801\n",
      "Early stopping, best iteration is:\n",
      "[90]\tvalid_0's multi_logloss: 0.37472\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.031196 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 2269\n",
      "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n",
      "[LightGBM] [Info] Start training from score -1.009534\n",
      "[LightGBM] [Info] Start training from score -0.717940\n",
      "[LightGBM] [Info] Start training from score -2.789075\n",
      "[LightGBM] [Info] Start training from score -5.347629\n",
      "[LightGBM] [Info] Start training from score -4.112309\n",
      "[LightGBM] [Info] Start training from score -3.508220\n",
      "[LightGBM] [Info] Start training from score -3.343939\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's multi_logloss: 0.442955\n",
      "[100]\tvalid_0's multi_logloss: 0.396755\n",
      "[150]\tvalid_0's multi_logloss: 0.378273\n",
      "Early stopping, best iteration is:\n",
      "[135]\tvalid_0's multi_logloss: 0.375671\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.018390 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 2260\n",
      "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n",
      "[LightGBM] [Info] Start training from score -1.009534\n",
      "[LightGBM] [Info] Start training from score -0.717940\n",
      "[LightGBM] [Info] Start training from score -2.789075\n",
      "[LightGBM] [Info] Start training from score -5.348131\n",
      "[LightGBM] [Info] Start training from score -4.112163\n",
      "[LightGBM] [Info] Start training from score -3.508220\n",
      "[LightGBM] [Info] Start training from score -3.343939\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's multi_logloss: 0.431872\n",
      "[100]\tvalid_0's multi_logloss: 0.378335\n",
      "Early stopping, best iteration is:\n",
      "[109]\tvalid_0's multi_logloss: 0.37075\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.032231 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 2262\n",
      "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n",
      "[LightGBM] [Info] Start training from score -1.009528\n",
      "[LightGBM] [Info] Start training from score -0.717945\n",
      "[LightGBM] [Info] Start training from score -2.789114\n",
      "[LightGBM] [Info] Start training from score -5.348131\n",
      "[LightGBM] [Info] Start training from score -4.112163\n",
      "[LightGBM] [Info] Start training from score -3.508220\n",
      "[LightGBM] [Info] Start training from score -3.343871\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's multi_logloss: 0.437033\n",
      "[100]\tvalid_0's multi_logloss: 0.387111\n",
      "Early stopping, best iteration is:\n",
      "[109]\tvalid_0's multi_logloss: 0.380009\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.031067 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 2268\n",
      "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n",
      "[LightGBM] [Info] Start training from score -1.009528\n",
      "[LightGBM] [Info] Start training from score -0.717945\n",
      "[LightGBM] [Info] Start training from score -2.789114\n",
      "[LightGBM] [Info] Start training from score -5.348131\n",
      "[LightGBM] [Info] Start training from score -4.112163\n",
      "[LightGBM] [Info] Start training from score -3.508220\n",
      "[LightGBM] [Info] Start training from score -3.343871\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's multi_logloss: 0.428647\n",
      "[100]\tvalid_0's multi_logloss: 0.485091\n",
      "Early stopping, best iteration is:\n",
      "[85]\tvalid_0's multi_logloss: 0.386918\n",
      "Wall time: 1min 53s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "test_predict = 0\n",
    "params = {\"boosting_type\": \"gbdt\",\n",
    "          \"objective\": \"multiclass\",\n",
    "          \"num_class': 7,\n",
    "          \"metric\": \"multi_logloss\"}\n",
    "for i, (trn_idx, val_idx) in enumerate(skf.split(train_x, train_y)):\n",
    "    dtrain = lgb.Dataset(train_x.iloc[trn_idx], label=train_y.iloc[trn_idx])\n",
    "    dvalid = lgb.Dataset(train_x.iloc[val_idx], label=train_y.iloc[val_idx])\n",
    "    model = lgb.train(params=params, train_set=dtrain,valid_sets=[dvalid],\n",
    "                      verbose_eval=50,\n",
    "                      early_stopping_rounds=20,\n",
    "                      num_boost_round=5000)\n",
    "    test_predict += model.predict(test_x)\n",
    "predict = test_predict/5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "6d7d4447",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "不使用optuna的多分类分数： 0.8631716636260369\n"
     ]
    }
   ],
   "source": [
    "# 评估指标acc，越大越好\n",
    "print('不使用optuna的多分类分数：', accuracy_score(np.argmax(predict, axis=1), test_y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "f6571309",
   "metadata": {},
   "outputs": [],
   "source": [
    "import optuna\n",
    "def objective(trial,train_x, train_y, valid_x, valid_y):\n",
    "    dtrain = lgb.Dataset(train_x, label=train_y)\n",
    "    dvalid = lgb.Dataset(valid_x, label=valid_y)\n",
    "\n",
    "    param = {\n",
    "        \"objective\": \"multiclass\",\n",
    "        \"metric\": \"multi_logloss\",\n",
    "        \"verbosity\": -1,\n",
    "        \"num_class\": 7,\n",
    "        \"boosting_type\": \"gbdt\",\n",
    "        'random_state':42,\n",
    "        \"lambda_l1\": trial.suggest_float(\"lambda_l1\", 1e-8, 10.0, log=True),\n",
    "        \"lambda_l2\": trial.suggest_float(\"lambda_l2\", 1e-8, 10.0, log=True),\n",
    "        \"num_leaves\": trial.suggest_int(\"num_leaves\", 2, 256),\n",
    "        \"feature_fraction\": trial.suggest_float(\"feature_fraction\", 0.4, 1.0),\n",
    "        \"bagging_fraction\": trial.suggest_float(\"bagging_fraction\", 0.4, 1.0),\n",
    "        \"bagging_freq\": trial.suggest_int(\"bagging_freq\", 1, 7),\n",
    "        \"min_child_samples\": trial.suggest_int(\"min_child_samples\", 5, 100),\n",
    "    }\n",
    "\n",
    "    # Add a callback for pruning.\n",
    "    pruning_callback = optuna.integration.LightGBMPruningCallback(trial, \"multi_logloss\")\n",
    "    gbm = lgb.train(\n",
    "        param, dtrain, valid_sets=[dvalid], verbose_eval=False, callbacks=[pruning_callback]\n",
    "    )\n",
    "\n",
    "    preds = gbm.predict(valid_x)\n",
    "    pred_labels = np.rint(preds)\n",
    "    acc = accuracy_score(np.argmax(pred_labels, axis=1), valid_y)\n",
    "    return acc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "4170d934",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 18:21:48,373]\u001b[0m A new study created in memory with name: no-name-7584aee7-d7a5-42ce-8426-7d4c7e034372\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:22:17,890]\u001b[0m Trial 0 finished with value: 0.8652253733912145 and parameters: {'lambda_l1': 0.00029350441808598135, 'lambda_l2': 0.0050140304157587625, 'num_leaves': 41, 'feature_fraction': 0.7195254545980898, 'bagging_fraction': 0.6783357374559799, 'bagging_freq': 6, 'min_child_samples': 32}. Best is trial 0 with value: 0.8652253733912145.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:22:40,277]\u001b[0m Trial 1 finished with value: 0.865770400260083 and parameters: {'lambda_l1': 8.073501039689362e-07, 'lambda_l2': 2.7820830665481296e-05, 'num_leaves': 41, 'feature_fraction': 0.8160512302334224, 'bagging_fraction': 0.6156688842906214, 'bagging_freq': 7, 'min_child_samples': 35}. Best is trial 0 with value: 0.8652253733912145.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:23:14,303]\u001b[0m Trial 2 finished with value: 0.9372358532060967 and parameters: {'lambda_l1': 1.1987047256838896e-07, 'lambda_l2': 0.0008429364945676234, 'num_leaves': 203, 'feature_fraction': 0.6245432353876832, 'bagging_fraction': 0.6680546579110309, 'bagging_freq': 1, 'min_child_samples': 91}. Best is trial 0 with value: 0.8652253733912145.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:23:37,365]\u001b[0m Trial 3 finished with value: 0.8423533686485246 and parameters: {'lambda_l1': 6.031700696477528e-05, 'lambda_l2': 0.24394450720600092, 'num_leaves': 33, 'feature_fraction': 0.48861458318031864, 'bagging_fraction': 0.9555014696451426, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:24:15,185]\u001b[0m Trial 4 finished with value: 0.9382876594442638 and parameters: {'lambda_l1': 0.013655390719191782, 'lambda_l2': 2.248487596614776e-07, 'num_leaves': 229, 'feature_fraction': 0.6314327078814898, 'bagging_fraction': 0.9426524187528247, 'bagging_freq': 1, 'min_child_samples': 72}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:24:18,728]\u001b[0m Trial 5 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:25:04,830]\u001b[0m Trial 6 finished with value: 0.9459084737335297 and parameters: {'lambda_l1': 0.02773561511163358, 'lambda_l2': 0.30726353465031164, 'num_leaves': 238, 'feature_fraction': 0.8197796474237233, 'bagging_fraction': 0.9951387365343576, 'bagging_freq': 7, 'min_child_samples': 75}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:25:10,692]\u001b[0m Trial 7 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:25:52,087]\u001b[0m Trial 8 finished with value: 0.9404582050448452 and parameters: {'lambda_l1': 2.969808205332883e-07, 'lambda_l2': 3.0108163042597293e-05, 'num_leaves': 184, 'feature_fraction': 0.9858004420758661, 'bagging_fraction': 0.7201379680285598, 'bagging_freq': 2, 'min_child_samples': 93}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:25:57,802]\u001b[0m Trial 9 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:26:02,460]\u001b[0m Trial 10 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:26:07,268]\u001b[0m Trial 11 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:26:11,252]\u001b[0m Trial 12 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:26:15,570]\u001b[0m Trial 13 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:26:21,086]\u001b[0m Trial 14 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:26:23,907]\u001b[0m Trial 15 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:26:27,922]\u001b[0m Trial 16 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:26:33,712]\u001b[0m Trial 17 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:26:37,313]\u001b[0m Trial 18 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:26:42,273]\u001b[0m Trial 19 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:26:46,859]\u001b[0m Trial 20 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:26:50,342]\u001b[0m Trial 21 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:26:55,710]\u001b[0m Trial 22 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:26:59,948]\u001b[0m Trial 23 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:27:03,848]\u001b[0m Trial 24 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:27:08,309]\u001b[0m Trial 25 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:27:13,458]\u001b[0m Trial 26 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:27:16,591]\u001b[0m Trial 27 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:27:21,016]\u001b[0m Trial 28 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:27:24,997]\u001b[0m Trial 29 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:27:32,574]\u001b[0m Trial 30 pruned. Trial was pruned at iteration 19.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:27:37,797]\u001b[0m Trial 31 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:27:43,562]\u001b[0m Trial 32 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:28:21,369]\u001b[0m Trial 33 finished with value: 0.9396167600543115 and parameters: {'lambda_l1': 1.7053800672111565e-08, 'lambda_l2': 0.009838404261194042, 'num_leaves': 205, 'feature_fraction': 0.8042641806396126, 'bagging_fraction': 0.4068986434462453, 'bagging_freq': 1, 'min_child_samples': 75}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:28:28,805]\u001b[0m Trial 34 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:28:32,440]\u001b[0m Trial 35 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:29:16,242]\u001b[0m Trial 36 finished with value: 0.9376948232009332 and parameters: {'lambda_l1': 0.00025741391284768883, 'lambda_l2': 1.3903821165130423e-05, 'num_leaves': 179, 'feature_fraction': 0.7513521003294499, 'bagging_fraction': 0.6877869004228008, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:29:22,290]\u001b[0m Trial 37 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:29:27,666]\u001b[0m Trial 38 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:29:33,842]\u001b[0m Trial 39 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:29:38,984]\u001b[0m Trial 40 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:30:13,915]\u001b[0m Trial 41 finished with value: 0.9370828632078178 and parameters: {'lambda_l1': 0.0003772109478239288, 'lambda_l2': 1.4609599162418084e-05, 'num_leaves': 177, 'feature_fraction': 0.7423326328000164, 'bagging_fraction': 0.6856961600540766, 'bagging_freq': 1, 'min_child_samples': 81}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:30:48,039]\u001b[0m Trial 42 finished with value: 0.9382972213191563 and parameters: {'lambda_l1': 0.0005140907001059531, 'lambda_l2': 2.4931963200658e-05, 'num_leaves': 191, 'feature_fraction': 0.7220599193293482, 'bagging_fraction': 0.6449224331658816, 'bagging_freq': 1, 'min_child_samples': 65}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:30:52,499]\u001b[0m Trial 43 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:31:27,381]\u001b[0m Trial 44 pruned. Trial was pruned at iteration 93.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:31:32,050]\u001b[0m Trial 45 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:32:04,026]\u001b[0m Trial 46 pruned. Trial was pruned at iteration 94.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:32:08,636]\u001b[0m Trial 47 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:32:14,065]\u001b[0m Trial 48 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:32:17,136]\u001b[0m Trial 49 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:32:22,164]\u001b[0m Trial 50 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:32:56,694]\u001b[0m Trial 51 finished with value: 0.9398653688015146 and parameters: {'lambda_l1': 8.007598044441421e-05, 'lambda_l2': 1.9006501098879968e-05, 'num_leaves': 193, 'feature_fraction': 0.7474479951356153, 'bagging_fraction': 0.6831044198923434, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 18:33:02,134]\u001b[0m Trial 52 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:33:25,476]\u001b[0m Trial 53 pruned. Trial was pruned at iteration 75.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:33:30,367]\u001b[0m Trial 54 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:33:34,302]\u001b[0m Trial 55 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:33:40,033]\u001b[0m Trial 56 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:33:43,381]\u001b[0m Trial 57 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:33:47,728]\u001b[0m Trial 58 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:33:52,240]\u001b[0m Trial 59 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:33:56,752]\u001b[0m Trial 60 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:34:02,449]\u001b[0m Trial 61 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:34:07,140]\u001b[0m Trial 62 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:34:13,623]\u001b[0m Trial 63 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:34:19,092]\u001b[0m Trial 64 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:34:57,096]\u001b[0m Trial 65 finished with value: 0.945717236235681 and parameters: {'lambda_l1': 0.0006215692281330095, 'lambda_l2': 0.0025507405649904265, 'num_leaves': 254, 'feature_fraction': 0.684182512525035, 'bagging_fraction': 0.881844626841578, 'bagging_freq': 1, 'min_child_samples': 74}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:35:04,313]\u001b[0m Trial 66 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:35:08,810]\u001b[0m Trial 67 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:35:13,882]\u001b[0m Trial 68 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:35:18,314]\u001b[0m Trial 69 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:35:21,402]\u001b[0m Trial 70 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:35:26,827]\u001b[0m Trial 71 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:36:06,776]\u001b[0m Trial 72 finished with value: 0.9417586200302155 and parameters: {'lambda_l1': 0.0008708841660641308, 'lambda_l2': 5.4043063920808084e-05, 'num_leaves': 224, 'feature_fraction': 0.7182774647861223, 'bagging_fraction': 0.6574176618253137, 'bagging_freq': 1, 'min_child_samples': 78}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:36:10,840]\u001b[0m Trial 73 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:36:16,482]\u001b[0m Trial 74 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:36:45,510]\u001b[0m Trial 75 pruned. Trial was pruned at iteration 85.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:36:50,598]\u001b[0m Trial 76 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:36:56,577]\u001b[0m Trial 77 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:37:01,576]\u001b[0m Trial 78 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:37:05,793]\u001b[0m Trial 79 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:37:41,797]\u001b[0m Trial 80 finished with value: 0.9436996806333786 and parameters: {'lambda_l1': 0.08921307105729769, 'lambda_l2': 4.975883016961999e-06, 'num_leaves': 214, 'feature_fraction': 0.8281286566426087, 'bagging_fraction': 0.5953173368434432, 'bagging_freq': 1, 'min_child_samples': 93}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:37:46,363]\u001b[0m Trial 81 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:37:51,749]\u001b[0m Trial 82 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:37:56,140]\u001b[0m Trial 83 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:38:00,762]\u001b[0m Trial 84 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:38:06,090]\u001b[0m Trial 85 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:38:10,580]\u001b[0m Trial 86 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:38:48,850]\u001b[0m Trial 87 finished with value: 0.9479260293358321 and parameters: {'lambda_l1': 0.02890836192523439, 'lambda_l2': 4.025111402953753e-05, 'num_leaves': 247, 'feature_fraction': 0.8727646045462183, 'bagging_fraction': 0.7437461253889492, 'bagging_freq': 1, 'min_child_samples': 83}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:38:54,311]\u001b[0m Trial 88 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:39:36,775]\u001b[0m Trial 89 finished with value: 0.9436136237593468 and parameters: {'lambda_l1': 2.776285425246586e-05, 'lambda_l2': 3.1180901688950096, 'num_leaves': 235, 'feature_fraction': 0.8371612462873237, 'bagging_fraction': 0.6753094794878766, 'bagging_freq': 2, 'min_child_samples': 92}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:39:41,160]\u001b[0m Trial 90 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:40:15,410]\u001b[0m Trial 91 finished with value: 0.9393968369317856 and parameters: {'lambda_l1': 7.819588207346975e-05, 'lambda_l2': 9.678552881824005e-06, 'num_leaves': 191, 'feature_fraction': 0.7476569605668175, 'bagging_fraction': 0.6665662519078225, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:40:20,661]\u001b[0m Trial 92 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:40:23,908]\u001b[0m Trial 93 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:40:30,385]\u001b[0m Trial 94 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:40:34,950]\u001b[0m Trial 95 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:40:41,586]\u001b[0m Trial 96 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:40:46,857]\u001b[0m Trial 97 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:40:49,830]\u001b[0m Trial 98 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:40:55,460]\u001b[0m Trial 99 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of finished trials: 100\n",
      "Best trial:\n",
      "  Value: 0.8423533686485246\n",
      "  Params: \n",
      "    lambda_l1: 6.031700696477528e-05\n",
      "    lambda_l2: 0.24394450720600092\n",
      "    num_leaves: 33\n",
      "    feature_fraction: 0.48861458318031864\n",
      "    bagging_fraction: 0.9555014696451426\n",
      "    bagging_freq: 1\n",
      "    min_child_samples: 79\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.023061 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 2262\n",
      "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 52\n",
      "[LightGBM] [Info] Start training from score -1.009534\n",
      "[LightGBM] [Info] Start training from score -0.717940\n",
      "[LightGBM] [Info] Start training from score -2.789075\n",
      "[LightGBM] [Info] Start training from score -5.347629\n",
      "[LightGBM] [Info] Start training from score -4.112163\n",
      "[LightGBM] [Info] Start training from score -3.508300\n",
      "[LightGBM] [Info] Start training from score -3.343939\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's multi_logloss: 0.444681\n",
      "[100]\tvalid_0's multi_logloss: 0.37701\n",
      "[150]\tvalid_0's multi_logloss: 0.33841\n",
      "[200]\tvalid_0's multi_logloss: 0.311971\n",
      "[250]\tvalid_0's multi_logloss: 0.291695\n",
      "[300]\tvalid_0's multi_logloss: 0.274889\n",
      "[350]\tvalid_0's multi_logloss: 0.261281\n",
      "[400]\tvalid_0's multi_logloss: 0.250771\n",
      "[450]\tvalid_0's multi_logloss: 0.240104\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[500]\tvalid_0's multi_logloss: 0.230589\n",
      "[550]\tvalid_0's multi_logloss: 0.222278\n",
      "[600]\tvalid_0's multi_logloss: 0.215255\n",
      "[650]\tvalid_0's multi_logloss: 0.208483\n",
      "[700]\tvalid_0's multi_logloss: 0.202692\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[750]\tvalid_0's multi_logloss: 0.196167\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[800]\tvalid_0's multi_logloss: 0.190411\n",
      "[850]\tvalid_0's multi_logloss: 0.185117\n",
      "[900]\tvalid_0's multi_logloss: 0.18105\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[950]\tvalid_0's multi_logloss: 0.176702\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1000]\tvalid_0's multi_logloss: 0.172928\n",
      "[1050]\tvalid_0's multi_logloss: 0.169457\n",
      "[1100]\tvalid_0's multi_logloss: 0.165919\n",
      "[1150]\tvalid_0's multi_logloss: 0.163006\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1200]\tvalid_0's multi_logloss: 0.15967\n",
      "[1250]\tvalid_0's multi_logloss: 0.156712\n",
      "[1300]\tvalid_0's multi_logloss: 0.154223\n",
      "[1350]\tvalid_0's multi_logloss: 0.151783\n",
      "[1400]\tvalid_0's multi_logloss: 0.149313\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1450]\tvalid_0's multi_logloss: 0.147362\n",
      "[1500]\tvalid_0's multi_logloss: 0.145167\n",
      "[1550]\tvalid_0's multi_logloss: 0.143229\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1600]\tvalid_0's multi_logloss: 0.141306\n",
      "[1650]\tvalid_0's multi_logloss: 0.139419\n",
      "[1700]\tvalid_0's multi_logloss: 0.137686\n",
      "[1750]\tvalid_0's multi_logloss: 0.136191\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1800]\tvalid_0's multi_logloss: 0.134525\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1850]\tvalid_0's multi_logloss: 0.13314\n",
      "[1900]\tvalid_0's multi_logloss: 0.131585\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1950]\tvalid_0's multi_logloss: 0.130162\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2000]\tvalid_0's multi_logloss: 0.128702\n",
      "[2050]\tvalid_0's multi_logloss: 0.127237\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2100]\tvalid_0's multi_logloss: 0.125969\n",
      "[2150]\tvalid_0's multi_logloss: 0.125018\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2200]\tvalid_0's multi_logloss: 0.124019\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2250]\tvalid_0's multi_logloss: 0.122787\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2300]\tvalid_0's multi_logloss: 0.121904\n",
      "[2350]\tvalid_0's multi_logloss: 0.120813\n",
      "[2400]\tvalid_0's multi_logloss: 0.119856\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2450]\tvalid_0's multi_logloss: 0.119105\n",
      "[2500]\tvalid_0's multi_logloss: 0.118181\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2550]\tvalid_0's multi_logloss: 0.117233\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2600]\tvalid_0's multi_logloss: 0.11653\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2650]\tvalid_0's multi_logloss: 0.115597\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2700]\tvalid_0's multi_logloss: 0.114851\n",
      "[2750]\tvalid_0's multi_logloss: 0.114109\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2800]\tvalid_0's multi_logloss: 0.113475\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2850]\tvalid_0's multi_logloss: 0.112871\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2900]\tvalid_0's multi_logloss: 0.112261\n",
      "[2950]\tvalid_0's multi_logloss: 0.111637\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3000]\tvalid_0's multi_logloss: 0.111155\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3050]\tvalid_0's multi_logloss: 0.110524\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3100]\tvalid_0's multi_logloss: 0.109987\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3150]\tvalid_0's multi_logloss: 0.1095\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3200]\tvalid_0's multi_logloss: 0.108922\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3250]\tvalid_0's multi_logloss: 0.108389\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3300]\tvalid_0's multi_logloss: 0.107919\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3350]\tvalid_0's multi_logloss: 0.107463\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3400]\tvalid_0's multi_logloss: 0.107071\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3450]\tvalid_0's multi_logloss: 0.106658\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3500]\tvalid_0's multi_logloss: 0.106334\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3550]\tvalid_0's multi_logloss: 0.105908\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3600]\tvalid_0's multi_logloss: 0.105497\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3650]\tvalid_0's multi_logloss: 0.105035\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3700]\tvalid_0's multi_logloss: 0.104725\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3750]\tvalid_0's multi_logloss: 0.104382\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3800]\tvalid_0's multi_logloss: 0.104083\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3850]\tvalid_0's multi_logloss: 0.103648\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3900]\tvalid_0's multi_logloss: 0.103237\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3950]\tvalid_0's multi_logloss: 0.102936\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4000]\tvalid_0's multi_logloss: 0.102547\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4050]\tvalid_0's multi_logloss: 0.102233\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4100]\tvalid_0's multi_logloss: 0.102018\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4150]\tvalid_0's multi_logloss: 0.10177\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4200]\tvalid_0's multi_logloss: 0.101442\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4250]\tvalid_0's multi_logloss: 0.101199\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4300]\tvalid_0's multi_logloss: 0.101029\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4350]\tvalid_0's multi_logloss: 0.100696\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4400]\tvalid_0's multi_logloss: 0.100538\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4450]\tvalid_0's multi_logloss: 0.100337\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4500]\tvalid_0's multi_logloss: 0.100081\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4550]\tvalid_0's multi_logloss: 0.0998246\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4600]\tvalid_0's multi_logloss: 0.0996299\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4650]\tvalid_0's multi_logloss: 0.0994822\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4700]\tvalid_0's multi_logloss: 0.0992758\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4750]\tvalid_0's multi_logloss: 0.0989493\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4800]\tvalid_0's multi_logloss: 0.098729\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4850]\tvalid_0's multi_logloss: 0.0984868\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4900]\tvalid_0's multi_logloss: 0.0983602\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4950]\tvalid_0's multi_logloss: 0.0982792\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[5000]\tvalid_0's multi_logloss: 0.0981563\n",
      "Did not meet early stopping. Best iteration is:\n",
      "[4998]\tvalid_0's multi_logloss: 0.098148\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 18:56:02,686]\u001b[0m Trial 100 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:56:09,367]\u001b[0m Trial 101 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:56:45,840]\u001b[0m Trial 102 finished with value: 0.9407450612916181 and parameters: {'lambda_l1': 8.197882698131674e-05, 'lambda_l2': 0.00021208640393700757, 'num_leaves': 192, 'feature_fraction': 0.7731692868777431, 'bagging_fraction': 0.6735311455374017, 'bagging_freq': 1, 'min_child_samples': 78}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:56:50,126]\u001b[0m Trial 103 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:57:26,832]\u001b[0m Trial 104 finished with value: 0.9414526400336578 and parameters: {'lambda_l1': 1.2795362377766212e-05, 'lambda_l2': 1.8844145747184915e-05, 'num_leaves': 217, 'feature_fraction': 0.7191746432394106, 'bagging_fraction': 0.7098988463324459, 'bagging_freq': 1, 'min_child_samples': 74}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:57:32,559]\u001b[0m Trial 105 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:57:37,952]\u001b[0m Trial 106 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:57:40,971]\u001b[0m Trial 107 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:57:45,702]\u001b[0m Trial 108 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:58:22,795]\u001b[0m Trial 109 finished with value: 0.942236713774837 and parameters: {'lambda_l1': 2.209464573982675e-06, 'lambda_l2': 8.287334662719688e-06, 'num_leaves': 207, 'feature_fraction': 0.7665224568657326, 'bagging_fraction': 0.6416358656740727, 'bagging_freq': 1, 'min_child_samples': 86}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:58:27,752]\u001b[0m Trial 110 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:58:32,799]\u001b[0m Trial 111 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:59:01,076]\u001b[0m Trial 112 pruned. Trial was pruned at iteration 88.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:59:06,071]\u001b[0m Trial 113 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:59:11,302]\u001b[0m Trial 114 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:59:15,597]\u001b[0m Trial 115 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:59:19,903]\u001b[0m Trial 116 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 18:59:33,007]\u001b[0m Trial 117 pruned. Trial was pruned at iteration 37.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:00:13,094]\u001b[0m Trial 118 finished with value: 0.9443212025013865 and parameters: {'lambda_l1': 0.002942524852825639, 'lambda_l2': 0.011610635505416795, 'num_leaves': 203, 'feature_fraction': 0.9531605764605278, 'bagging_fraction': 0.630189454389572, 'bagging_freq': 4, 'min_child_samples': 42}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:00:17,760]\u001b[0m Trial 119 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:00:23,215]\u001b[0m Trial 120 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:00:57,349]\u001b[0m Trial 121 finished with value: 0.9405347000439847 and parameters: {'lambda_l1': 7.143416155152381e-05, 'lambda_l2': 0.00017466919367877497, 'num_leaves': 193, 'feature_fraction': 0.7734033889080848, 'bagging_fraction': 0.6713509524709522, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:01:33,315]\u001b[0m Trial 122 finished with value: 0.9424757606471477 and parameters: {'lambda_l1': 3.671159464706789e-05, 'lambda_l2': 3.9461792591530047e-05, 'num_leaves': 209, 'feature_fraction': 0.7582936549212366, 'bagging_fraction': 0.6671570260041155, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:01:39,469]\u001b[0m Trial 123 pruned. Trial was pruned at iteration 12.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:02:15,641]\u001b[0m Trial 124 finished with value: 0.9422271518999445 and parameters: {'lambda_l1': 9.126230817825131e-08, 'lambda_l2': 9.42798578840402e-06, 'num_leaves': 212, 'feature_fraction': 0.7770330244830586, 'bagging_fraction': 0.6480274535778553, 'bagging_freq': 1, 'min_child_samples': 82}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:02:20,777]\u001b[0m Trial 125 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:02:26,240]\u001b[0m Trial 126 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:02:45,146]\u001b[0m Trial 127 pruned. Trial was pruned at iteration 61.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:02:50,386]\u001b[0m Trial 128 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:02:53,658]\u001b[0m Trial 129 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:02:58,360]\u001b[0m Trial 130 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:03:15,101]\u001b[0m Trial 131 pruned. Trial was pruned at iteration 50.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:03:50,148]\u001b[0m Trial 132 finished with value: 0.9417873056548928 and parameters: {'lambda_l1': 8.072026862454635e-05, 'lambda_l2': 8.510873972830464e-05, 'num_leaves': 201, 'feature_fraction': 0.7644190714627204, 'bagging_fraction': 0.679994557223264, 'bagging_freq': 1, 'min_child_samples': 77}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:03:54,573]\u001b[0m Trial 133 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:03:59,616]\u001b[0m Trial 134 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:04:03,756]\u001b[0m Trial 135 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:04:07,932]\u001b[0m Trial 136 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:04:14,765]\u001b[0m Trial 137 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:04:19,167]\u001b[0m Trial 138 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:04:54,001]\u001b[0m Trial 139 finished with value: 0.942370580023331 and parameters: {'lambda_l1': 3.232550131286934e-06, 'lambda_l2': 0.0006916472301971968, 'num_leaves': 202, 'feature_fraction': 0.8343548844441779, 'bagging_fraction': 0.8476577552042586, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:04:58,456]\u001b[0m Trial 140 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:05:04,372]\u001b[0m Trial 141 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:05:41,758]\u001b[0m Trial 142 finished with value: 0.9466256143504619 and parameters: {'lambda_l1': 2.980896851114429e-05, 'lambda_l2': 2.913417349261647e-05, 'num_leaves': 239, 'feature_fraction': 0.763135869473294, 'bagging_fraction': 0.7513284296958156, 'bagging_freq': 1, 'min_child_samples': 62}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:05:46,991]\u001b[0m Trial 143 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:05:53,817]\u001b[0m Trial 144 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:06:30,154]\u001b[0m Trial 145 finished with value: 0.9437474900078408 and parameters: {'lambda_l1': 7.212460033828516e-05, 'lambda_l2': 4.309094661715439e-06, 'num_leaves': 212, 'feature_fraction': 0.7533850653260141, 'bagging_fraction': 0.8916021652632435, 'bagging_freq': 1, 'min_child_samples': 75}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:07:06,486]\u001b[0m Trial 146 finished with value: 0.9439291656307969 and parameters: {'lambda_l1': 0.0005892869464862208, 'lambda_l2': 1.1242672082140708e-05, 'num_leaves': 223, 'feature_fraction': 0.7312666460187671, 'bagging_fraction': 0.6736774370139369, 'bagging_freq': 1, 'min_child_samples': 77}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:07:10,387]\u001b[0m Trial 147 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:07:14,676]\u001b[0m Trial 148 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:07:20,110]\u001b[0m Trial 149 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:07:24,141]\u001b[0m Trial 150 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:07:28,653]\u001b[0m Trial 151 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:08:04,981]\u001b[0m Trial 152 finished with value: 0.9427626168939206 and parameters: {'lambda_l1': 0.0011892669983091146, 'lambda_l2': 5.820028108882431e-05, 'num_leaves': 224, 'feature_fraction': 0.7351610234838635, 'bagging_fraction': 0.6511917047811185, 'bagging_freq': 1, 'min_child_samples': 81}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 19:08:41,831]\u001b[0m Trial 153 finished with value: 0.9482989424566369 and parameters: {'lambda_l1': 0.00037447863194847096, 'lambda_l2': 2.1542282896260357e-05, 'num_leaves': 234, 'feature_fraction': 0.9144698353599378, 'bagging_fraction': 0.6832488195447302, 'bagging_freq': 1, 'min_child_samples': 72}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:08:47,021]\u001b[0m Trial 154 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:08:52,838]\u001b[0m Trial 155 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:08:58,447]\u001b[0m Trial 156 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:09:02,729]\u001b[0m Trial 157 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:09:09,544]\u001b[0m Trial 158 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:09:13,126]\u001b[0m Trial 159 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:09:40,786]\u001b[0m Trial 160 pruned. Trial was pruned at iteration 88.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:09:56,557]\u001b[0m Trial 161 pruned. Trial was pruned at iteration 42.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:10:01,280]\u001b[0m Trial 162 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:10:37,093]\u001b[0m Trial 163 finished with value: 0.9425331318965022 and parameters: {'lambda_l1': 6.397677155805411e-08, 'lambda_l2': 0.00039621691928490413, 'num_leaves': 206, 'feature_fraction': 0.7667580883039189, 'bagging_fraction': 0.657903822017336, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:10:41,857]\u001b[0m Trial 164 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:10:47,583]\u001b[0m Trial 165 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:10:52,424]\u001b[0m Trial 166 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:10:58,913]\u001b[0m Trial 167 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:11:03,909]\u001b[0m Trial 168 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:11:09,381]\u001b[0m Trial 169 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:11:13,686]\u001b[0m Trial 170 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:11:49,897]\u001b[0m Trial 171 finished with value: 0.9430112256411237 and parameters: {'lambda_l1': 1.102134869211263e-07, 'lambda_l2': 7.503180286760575e-06, 'num_leaves': 214, 'feature_fraction': 0.783389889391717, 'bagging_fraction': 0.6498041172731315, 'bagging_freq': 1, 'min_child_samples': 82}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:12:26,420]\u001b[0m Trial 172 finished with value: 0.9431355300147253 and parameters: {'lambda_l1': 1.0270752816234116e-07, 'lambda_l2': 1.0817935884223942e-05, 'num_leaves': 211, 'feature_fraction': 0.7583412868303638, 'bagging_fraction': 0.6795874395289168, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:13:04,383]\u001b[0m Trial 173 finished with value: 0.9439004800061196 and parameters: {'lambda_l1': 8.279755593895644e-08, 'lambda_l2': 3.6313073067998552e-06, 'num_leaves': 230, 'feature_fraction': 0.7727565832075506, 'bagging_fraction': 0.6452464868388876, 'bagging_freq': 1, 'min_child_samples': 83}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:13:10,975]\u001b[0m Trial 174 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:13:46,828]\u001b[0m Trial 175 finished with value: 0.9445315637490199 and parameters: {'lambda_l1': 0.0002353217979236286, 'lambda_l2': 3.171263348326624e-05, 'num_leaves': 218, 'feature_fraction': 0.8203542425794451, 'bagging_fraction': 0.6588367468604717, 'bagging_freq': 1, 'min_child_samples': 76}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:13:51,827]\u001b[0m Trial 176 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:13:55,900]\u001b[0m Trial 177 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:14:32,835]\u001b[0m Trial 178 finished with value: 0.9489778355739994 and parameters: {'lambda_l1': 1.0560192692698665e-08, 'lambda_l2': 0.0002765136930724561, 'num_leaves': 246, 'feature_fraction': 0.7669065013864841, 'bagging_fraction': 0.6258778389490884, 'bagging_freq': 1, 'min_child_samples': 25}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:15:08,531]\u001b[0m Trial 179 finished with value: 0.9418638006540322 and parameters: {'lambda_l1': 5.8099076285804865e-08, 'lambda_l2': 9.604379213115968e-06, 'num_leaves': 208, 'feature_fraction': 0.7900315064460727, 'bagging_fraction': 0.6778057274702293, 'bagging_freq': 1, 'min_child_samples': 86}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:15:14,384]\u001b[0m Trial 180 pruned. Trial was pruned at iteration 12.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:15:22,172]\u001b[0m Trial 181 pruned. Trial was pruned at iteration 21.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:15:32,935]\u001b[0m Trial 182 pruned. Trial was pruned at iteration 32.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:16:09,350]\u001b[0m Trial 183 finished with value: 0.9439865368801514 and parameters: {'lambda_l1': 7.039591924287473e-08, 'lambda_l2': 2.5146082118354423e-05, 'num_leaves': 221, 'feature_fraction': 0.7397390119318659, 'bagging_fraction': 0.672737210757752, 'bagging_freq': 1, 'min_child_samples': 82}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:16:13,980]\u001b[0m Trial 184 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:16:18,553]\u001b[0m Trial 185 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:16:55,502]\u001b[0m Trial 186 finished with value: 0.9441682125031076 and parameters: {'lambda_l1': 1.9294698699801913e-07, 'lambda_l2': 2.5570868408037523e-06, 'num_leaves': 216, 'feature_fraction': 0.7829248911740091, 'bagging_fraction': 0.9993734316792878, 'bagging_freq': 1, 'min_child_samples': 78}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:17:01,011]\u001b[0m Trial 187 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:17:04,368]\u001b[0m Trial 188 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:17:09,352]\u001b[0m Trial 189 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:17:15,404]\u001b[0m Trial 190 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:17:20,157]\u001b[0m Trial 191 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:17:56,849]\u001b[0m Trial 192 finished with value: 0.9449427243693943 and parameters: {'lambda_l1': 3.0670346509523553e-06, 'lambda_l2': 7.037461956773899e-06, 'num_leaves': 226, 'feature_fraction': 0.7927923736502508, 'bagging_fraction': 0.6369345538445389, 'bagging_freq': 1, 'min_child_samples': 85}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:18:02,512]\u001b[0m Trial 193 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:18:06,965]\u001b[0m Trial 194 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:18:42,532]\u001b[0m Trial 195 finished with value: 0.941882924403817 and parameters: {'lambda_l1': 5.448535054846655e-05, 'lambda_l2': 0.008712731969886423, 'num_leaves': 211, 'feature_fraction': 0.7730826357400354, 'bagging_fraction': 0.6634899720026096, 'bagging_freq': 1, 'min_child_samples': 90}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:19:18,834]\u001b[0m Trial 196 finished with value: 0.9436040618844543 and parameters: {'lambda_l1': 6.64569708412817e-05, 'lambda_l2': 0.0027368345553249167, 'num_leaves': 212, 'feature_fraction': 0.8161286103449696, 'bagging_fraction': 0.6899947578536335, 'bagging_freq': 1, 'min_child_samples': 91}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:19:24,121]\u001b[0m Trial 197 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:19:28,683]\u001b[0m Trial 198 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:19:33,772]\u001b[0m Trial 199 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of finished trials: 200\n",
      "Best trial:\n",
      "  Value: 0.8423533686485246\n",
      "  Params: \n",
      "    lambda_l1: 6.031700696477528e-05\n",
      "    lambda_l2: 0.24394450720600092\n",
      "    num_leaves: 33\n",
      "    feature_fraction: 0.48861458318031864\n",
      "    bagging_fraction: 0.9555014696451426\n",
      "    bagging_freq: 1\n",
      "    min_child_samples: 79\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.027211 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 2267\n",
      "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 52\n",
      "[LightGBM] [Info] Start training from score -1.009534\n",
      "[LightGBM] [Info] Start training from score -0.717940\n",
      "[LightGBM] [Info] Start training from score -2.789075\n",
      "[LightGBM] [Info] Start training from score -5.347629\n",
      "[LightGBM] [Info] Start training from score -4.112309\n",
      "[LightGBM] [Info] Start training from score -3.508220\n",
      "[LightGBM] [Info] Start training from score -3.343939\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's multi_logloss: 0.442887\n",
      "[100]\tvalid_0's multi_logloss: 0.373911\n",
      "[150]\tvalid_0's multi_logloss: 0.336668\n",
      "[200]\tvalid_0's multi_logloss: 0.312308\n",
      "[250]\tvalid_0's multi_logloss: 0.293877\n",
      "[300]\tvalid_0's multi_logloss: 0.277526\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[350]\tvalid_0's multi_logloss: 0.262446\n",
      "[400]\tvalid_0's multi_logloss: 0.251811\n",
      "[450]\tvalid_0's multi_logloss: 0.240068\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[500]\tvalid_0's multi_logloss: 0.230934\n",
      "[550]\tvalid_0's multi_logloss: 0.222786\n",
      "[600]\tvalid_0's multi_logloss: 0.214897\n",
      "[650]\tvalid_0's multi_logloss: 0.207044\n",
      "[700]\tvalid_0's multi_logloss: 0.200602\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[750]\tvalid_0's multi_logloss: 0.194382\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[800]\tvalid_0's multi_logloss: 0.189488\n",
      "[850]\tvalid_0's multi_logloss: 0.185057\n",
      "[900]\tvalid_0's multi_logloss: 0.180852\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[950]\tvalid_0's multi_logloss: 0.17619\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1000]\tvalid_0's multi_logloss: 0.172508\n",
      "[1050]\tvalid_0's multi_logloss: 0.169138\n",
      "[1100]\tvalid_0's multi_logloss: 0.165975\n",
      "[1150]\tvalid_0's multi_logloss: 0.162635\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1200]\tvalid_0's multi_logloss: 0.15997\n",
      "[1250]\tvalid_0's multi_logloss: 0.156712\n",
      "[1300]\tvalid_0's multi_logloss: 0.1538\n",
      "[1350]\tvalid_0's multi_logloss: 0.151426\n",
      "[1400]\tvalid_0's multi_logloss: 0.149565\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1450]\tvalid_0's multi_logloss: 0.147572\n",
      "[1500]\tvalid_0's multi_logloss: 0.14554\n",
      "[1550]\tvalid_0's multi_logloss: 0.143424\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1600]\tvalid_0's multi_logloss: 0.141495\n",
      "[1650]\tvalid_0's multi_logloss: 0.139469\n",
      "[1700]\tvalid_0's multi_logloss: 0.137832\n",
      "[1750]\tvalid_0's multi_logloss: 0.13608\n",
      "[1800]\tvalid_0's multi_logloss: 0.134508\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1850]\tvalid_0's multi_logloss: 0.132867\n",
      "[1900]\tvalid_0's multi_logloss: 0.131194\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1950]\tvalid_0's multi_logloss: 0.1297\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2000]\tvalid_0's multi_logloss: 0.128321\n",
      "[2050]\tvalid_0's multi_logloss: 0.127052\n",
      "[2100]\tvalid_0's multi_logloss: 0.125839\n",
      "[2150]\tvalid_0's multi_logloss: 0.124789\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2200]\tvalid_0's multi_logloss: 0.123598\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2250]\tvalid_0's multi_logloss: 0.122598\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2300]\tvalid_0's multi_logloss: 0.121569\n",
      "[2350]\tvalid_0's multi_logloss: 0.120579\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2400]\tvalid_0's multi_logloss: 0.119715\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2450]\tvalid_0's multi_logloss: 0.118919\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2500]\tvalid_0's multi_logloss: 0.118045\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2550]\tvalid_0's multi_logloss: 0.117234\n",
      "[2600]\tvalid_0's multi_logloss: 0.116567\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2650]\tvalid_0's multi_logloss: 0.115834\n",
      "[2700]\tvalid_0's multi_logloss: 0.115171\n",
      "[2750]\tvalid_0's multi_logloss: 0.114451\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2800]\tvalid_0's multi_logloss: 0.11376\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2850]\tvalid_0's multi_logloss: 0.113166\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2900]\tvalid_0's multi_logloss: 0.112467\n",
      "[2950]\tvalid_0's multi_logloss: 0.111564\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3000]\tvalid_0's multi_logloss: 0.111056\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3050]\tvalid_0's multi_logloss: 0.110477\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3100]\tvalid_0's multi_logloss: 0.109703\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3150]\tvalid_0's multi_logloss: 0.109081\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3200]\tvalid_0's multi_logloss: 0.108637\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3250]\tvalid_0's multi_logloss: 0.108172\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3300]\tvalid_0's multi_logloss: 0.107737\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3350]\tvalid_0's multi_logloss: 0.107314\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3400]\tvalid_0's multi_logloss: 0.106692\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3450]\tvalid_0's multi_logloss: 0.106282\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3500]\tvalid_0's multi_logloss: 0.105917\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3550]\tvalid_0's multi_logloss: 0.105631\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3600]\tvalid_0's multi_logloss: 0.105321\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3650]\tvalid_0's multi_logloss: 0.104916\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3700]\tvalid_0's multi_logloss: 0.104489\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3750]\tvalid_0's multi_logloss: 0.104121\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3800]\tvalid_0's multi_logloss: 0.103784\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3850]\tvalid_0's multi_logloss: 0.103477\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3900]\tvalid_0's multi_logloss: 0.103191\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3950]\tvalid_0's multi_logloss: 0.102865\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4000]\tvalid_0's multi_logloss: 0.102531\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4050]\tvalid_0's multi_logloss: 0.102279\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4100]\tvalid_0's multi_logloss: 0.102028\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4150]\tvalid_0's multi_logloss: 0.101774\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4200]\tvalid_0's multi_logloss: 0.101443\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4250]\tvalid_0's multi_logloss: 0.101177\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4300]\tvalid_0's multi_logloss: 0.101004\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4350]\tvalid_0's multi_logloss: 0.100807\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4400]\tvalid_0's multi_logloss: 0.100584\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4450]\tvalid_0's multi_logloss: 0.100384\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4500]\tvalid_0's multi_logloss: 0.100211\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4550]\tvalid_0's multi_logloss: 0.0999913\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4600]\tvalid_0's multi_logloss: 0.0998812\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4650]\tvalid_0's multi_logloss: 0.0997132\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4700]\tvalid_0's multi_logloss: 0.0994868\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4750]\tvalid_0's multi_logloss: 0.0992342\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4800]\tvalid_0's multi_logloss: 0.0989926\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4850]\tvalid_0's multi_logloss: 0.0987583\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "Early stopping, best iteration is:\n",
      "[4860]\tvalid_0's multi_logloss: 0.0987031\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 19:35:44,152]\u001b[0m Trial 200 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:35:51,579]\u001b[0m Trial 201 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:35:57,380]\u001b[0m Trial 202 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:36:03,314]\u001b[0m Trial 203 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:36:07,983]\u001b[0m Trial 204 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:36:13,613]\u001b[0m Trial 205 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:36:21,694]\u001b[0m Trial 206 pruned. Trial was pruned at iteration 21.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:36:25,647]\u001b[0m Trial 207 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:36:30,111]\u001b[0m Trial 208 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:36:37,233]\u001b[0m Trial 209 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:36:42,081]\u001b[0m Trial 210 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:36:49,029]\u001b[0m Trial 211 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:36:53,527]\u001b[0m Trial 212 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:37:00,135]\u001b[0m Trial 213 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:37:04,820]\u001b[0m Trial 214 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:37:09,463]\u001b[0m Trial 215 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:37:13,289]\u001b[0m Trial 216 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:37:16,788]\u001b[0m Trial 217 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:37:23,053]\u001b[0m Trial 218 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:37:59,676]\u001b[0m Trial 219 finished with value: 0.9434319481363905 and parameters: {'lambda_l1': 1.1871087780899598e-07, 'lambda_l2': 0.00023968457905386466, 'num_leaves': 219, 'feature_fraction': 0.838039247899186, 'bagging_fraction': 0.6772774131242125, 'bagging_freq': 1, 'min_child_samples': 81}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:38:04,526]\u001b[0m Trial 220 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:38:41,652]\u001b[0m Trial 221 finished with value: 0.9425618175211795 and parameters: {'lambda_l1': 8.509891041969528e-05, 'lambda_l2': 3.581606389921117e-05, 'num_leaves': 209, 'feature_fraction': 0.8843923083599923, 'bagging_fraction': 0.6707209079942898, 'bagging_freq': 1, 'min_child_samples': 80}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:38:50,823]\u001b[0m Trial 222 pruned. Trial was pruned at iteration 22.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:38:55,262]\u001b[0m Trial 223 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:39:00,904]\u001b[0m Trial 224 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:39:06,074]\u001b[0m Trial 225 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:39:10,767]\u001b[0m Trial 226 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:39:15,341]\u001b[0m Trial 227 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:39:19,652]\u001b[0m Trial 228 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:39:57,821]\u001b[0m Trial 229 finished with value: 0.9456311793616492 and parameters: {'lambda_l1': 0.000357294778794299, 'lambda_l2': 4.383896339017896e-05, 'num_leaves': 218, 'feature_fraction': 0.9990787264144749, 'bagging_fraction': 0.6859577686240965, 'bagging_freq': 1, 'min_child_samples': 96}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:40:03,271]\u001b[0m Trial 230 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:40:07,882]\u001b[0m Trial 231 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:40:12,972]\u001b[0m Trial 232 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:40:21,168]\u001b[0m Trial 233 pruned. Trial was pruned at iteration 22.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:40:26,938]\u001b[0m Trial 234 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:40:30,734]\u001b[0m Trial 235 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:40:35,341]\u001b[0m Trial 236 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:40:41,601]\u001b[0m Trial 237 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:40:46,145]\u001b[0m Trial 238 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:40:51,441]\u001b[0m Trial 239 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:40:56,059]\u001b[0m Trial 240 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:41:01,106]\u001b[0m Trial 241 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:41:06,655]\u001b[0m Trial 242 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:41:11,262]\u001b[0m Trial 243 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:41:48,687]\u001b[0m Trial 244 finished with value: 0.9444550687498805 and parameters: {'lambda_l1': 0.0002036623168757251, 'lambda_l2': 0.006006535254678557, 'num_leaves': 215, 'feature_fraction': 0.9253229899202093, 'bagging_fraction': 0.6880698265447902, 'bagging_freq': 1, 'min_child_samples': 78}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:41:54,485]\u001b[0m Trial 245 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:41:58,496]\u001b[0m Trial 246 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:42:05,675]\u001b[0m Trial 247 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:42:16,623]\u001b[0m Trial 248 pruned. Trial was pruned at iteration 27.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:42:21,269]\u001b[0m Trial 249 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:42:26,960]\u001b[0m Trial 250 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:42:33,177]\u001b[0m Trial 251 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:42:39,276]\u001b[0m Trial 252 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:42:43,691]\u001b[0m Trial 253 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:42:48,271]\u001b[0m Trial 254 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:42:54,532]\u001b[0m Trial 255 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:42:58,714]\u001b[0m Trial 256 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:43:05,669]\u001b[0m Trial 257 pruned. Trial was pruned at iteration 12.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:43:09,875]\u001b[0m Trial 258 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:43:15,085]\u001b[0m Trial 259 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:43:20,144]\u001b[0m Trial 260 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:43:23,667]\u001b[0m Trial 261 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:43:30,759]\u001b[0m Trial 262 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:43:34,913]\u001b[0m Trial 263 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:43:40,565]\u001b[0m Trial 264 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:43:47,697]\u001b[0m Trial 265 pruned. Trial was pruned at iteration 17.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:43:53,115]\u001b[0m Trial 266 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:43:57,659]\u001b[0m Trial 267 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:44:02,170]\u001b[0m Trial 268 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:44:07,650]\u001b[0m Trial 269 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:44:12,015]\u001b[0m Trial 270 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:44:21,808]\u001b[0m Trial 271 pruned. Trial was pruned at iteration 25.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:44:26,653]\u001b[0m Trial 272 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:44:32,936]\u001b[0m Trial 273 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:44:36,245]\u001b[0m Trial 274 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:44:41,807]\u001b[0m Trial 275 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:44:46,993]\u001b[0m Trial 276 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 19:44:52,300]\u001b[0m Trial 277 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:44:57,150]\u001b[0m Trial 278 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:45:34,096]\u001b[0m Trial 279 finished with value: 0.9436996806333786 and parameters: {'lambda_l1': 0.00031112512112930536, 'lambda_l2': 0.0001880477331732333, 'num_leaves': 220, 'feature_fraction': 0.852594454408188, 'bagging_fraction': 0.6470457333056485, 'bagging_freq': 1, 'min_child_samples': 79}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:45:38,422]\u001b[0m Trial 280 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:46:15,718]\u001b[0m Trial 281 finished with value: 0.9430685968904783 and parameters: {'lambda_l1': 4.45254298816343e-08, 'lambda_l2': 0.001188876991401564, 'num_leaves': 227, 'feature_fraction': 0.7761533930785017, 'bagging_fraction': 0.6795762898644894, 'bagging_freq': 1, 'min_child_samples': 74}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:46:21,752]\u001b[0m Trial 282 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:46:26,097]\u001b[0m Trial 283 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:46:31,317]\u001b[0m Trial 284 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:46:36,020]\u001b[0m Trial 285 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:46:41,005]\u001b[0m Trial 286 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:46:48,527]\u001b[0m Trial 287 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:46:54,368]\u001b[0m Trial 288 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:46:59,084]\u001b[0m Trial 289 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:47:03,641]\u001b[0m Trial 290 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:47:09,582]\u001b[0m Trial 291 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:47:13,893]\u001b[0m Trial 292 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:47:33,116]\u001b[0m Trial 293 pruned. Trial was pruned at iteration 57.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:47:36,632]\u001b[0m Trial 294 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:47:41,787]\u001b[0m Trial 295 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:47:46,895]\u001b[0m Trial 296 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:47:49,470]\u001b[0m Trial 297 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:47:54,857]\u001b[0m Trial 298 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 19:47:58,624]\u001b[0m Trial 299 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of finished trials: 300\n",
      "Best trial:\n",
      "  Value: 0.8423533686485246\n",
      "  Params: \n",
      "    lambda_l1: 6.031700696477528e-05\n",
      "    lambda_l2: 0.24394450720600092\n",
      "    num_leaves: 33\n",
      "    feature_fraction: 0.48861458318031864\n",
      "    bagging_fraction: 0.9555014696451426\n",
      "    bagging_freq: 1\n",
      "    min_child_samples: 79\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.026580 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 2258\n",
      "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 52\n",
      "[LightGBM] [Info] Start training from score -1.009534\n",
      "[LightGBM] [Info] Start training from score -0.717940\n",
      "[LightGBM] [Info] Start training from score -2.789075\n",
      "[LightGBM] [Info] Start training from score -5.348131\n",
      "[LightGBM] [Info] Start training from score -4.112163\n",
      "[LightGBM] [Info] Start training from score -3.508220\n",
      "[LightGBM] [Info] Start training from score -3.343939\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's multi_logloss: 0.445843\n",
      "[100]\tvalid_0's multi_logloss: 0.377834\n",
      "[150]\tvalid_0's multi_logloss: 0.341664\n",
      "[200]\tvalid_0's multi_logloss: 0.316826\n",
      "[250]\tvalid_0's multi_logloss: 0.296451\n",
      "[300]\tvalid_0's multi_logloss: 0.281169\n",
      "[350]\tvalid_0's multi_logloss: 0.265537\n",
      "[400]\tvalid_0's multi_logloss: 0.254785\n",
      "[450]\tvalid_0's multi_logloss: 0.243631\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[500]\tvalid_0's multi_logloss: 0.234101\n",
      "[550]\tvalid_0's multi_logloss: 0.226309\n",
      "[600]\tvalid_0's multi_logloss: 0.21796\n",
      "[650]\tvalid_0's multi_logloss: 0.211559\n",
      "[700]\tvalid_0's multi_logloss: 0.204727\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[750]\tvalid_0's multi_logloss: 0.198816\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[800]\tvalid_0's multi_logloss: 0.193023\n",
      "[850]\tvalid_0's multi_logloss: 0.188199\n",
      "[900]\tvalid_0's multi_logloss: 0.184119\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[950]\tvalid_0's multi_logloss: 0.180033\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1000]\tvalid_0's multi_logloss: 0.176498\n",
      "[1050]\tvalid_0's multi_logloss: 0.172741\n",
      "[1100]\tvalid_0's multi_logloss: 0.169305\n",
      "[1150]\tvalid_0's multi_logloss: 0.166002\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1200]\tvalid_0's multi_logloss: 0.162865\n",
      "[1250]\tvalid_0's multi_logloss: 0.159812\n",
      "[1300]\tvalid_0's multi_logloss: 0.157137\n",
      "[1350]\tvalid_0's multi_logloss: 0.154434\n",
      "[1400]\tvalid_0's multi_logloss: 0.151964\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1450]\tvalid_0's multi_logloss: 0.149897\n",
      "[1500]\tvalid_0's multi_logloss: 0.14801\n",
      "[1550]\tvalid_0's multi_logloss: 0.146389\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1600]\tvalid_0's multi_logloss: 0.144423\n",
      "[1650]\tvalid_0's multi_logloss: 0.142452\n",
      "[1700]\tvalid_0's multi_logloss: 0.140879\n",
      "[1750]\tvalid_0's multi_logloss: 0.138966\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1800]\tvalid_0's multi_logloss: 0.137507\n",
      "[1850]\tvalid_0's multi_logloss: 0.135709\n",
      "[1900]\tvalid_0's multi_logloss: 0.134087\n",
      "[1950]\tvalid_0's multi_logloss: 0.13268\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2000]\tvalid_0's multi_logloss: 0.13141\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2050]\tvalid_0's multi_logloss: 0.130194\n",
      "[2100]\tvalid_0's multi_logloss: 0.129083\n",
      "[2150]\tvalid_0's multi_logloss: 0.127913\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2200]\tvalid_0's multi_logloss: 0.126733\n",
      "[2250]\tvalid_0's multi_logloss: 0.125701\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2300]\tvalid_0's multi_logloss: 0.124613\n",
      "[2350]\tvalid_0's multi_logloss: 0.12328\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2400]\tvalid_0's multi_logloss: 0.122431\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2450]\tvalid_0's multi_logloss: 0.121605\n",
      "[2500]\tvalid_0's multi_logloss: 0.120838\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2550]\tvalid_0's multi_logloss: 0.119876\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2600]\tvalid_0's multi_logloss: 0.119193\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2650]\tvalid_0's multi_logloss: 0.118348\n",
      "[2700]\tvalid_0's multi_logloss: 0.117563\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2750]\tvalid_0's multi_logloss: 0.116756\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2800]\tvalid_0's multi_logloss: 0.116241\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2850]\tvalid_0's multi_logloss: 0.11561\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2900]\tvalid_0's multi_logloss: 0.115103\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2950]\tvalid_0's multi_logloss: 0.114529\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3000]\tvalid_0's multi_logloss: 0.113988\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3050]\tvalid_0's multi_logloss: 0.113383\n",
      "[3100]\tvalid_0's multi_logloss: 0.112689\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3150]\tvalid_0's multi_logloss: 0.112138\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3200]\tvalid_0's multi_logloss: 0.111635\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3250]\tvalid_0's multi_logloss: 0.111011\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3300]\tvalid_0's multi_logloss: 0.110546\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3350]\tvalid_0's multi_logloss: 0.110069\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3400]\tvalid_0's multi_logloss: 0.109636\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3450]\tvalid_0's multi_logloss: 0.109215\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3500]\tvalid_0's multi_logloss: 0.108815\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3550]\tvalid_0's multi_logloss: 0.108466\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3600]\tvalid_0's multi_logloss: 0.10805\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3650]\tvalid_0's multi_logloss: 0.107592\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3700]\tvalid_0's multi_logloss: 0.107273\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3750]\tvalid_0's multi_logloss: 0.106966\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3800]\tvalid_0's multi_logloss: 0.106652\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3850]\tvalid_0's multi_logloss: 0.106292\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3900]\tvalid_0's multi_logloss: 0.105884\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3950]\tvalid_0's multi_logloss: 0.105564\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4000]\tvalid_0's multi_logloss: 0.105233\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4050]\tvalid_0's multi_logloss: 0.104994\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4100]\tvalid_0's multi_logloss: 0.104723\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4150]\tvalid_0's multi_logloss: 0.104496\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4200]\tvalid_0's multi_logloss: 0.104218\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4250]\tvalid_0's multi_logloss: 0.103959\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4300]\tvalid_0's multi_logloss: 0.103701\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4350]\tvalid_0's multi_logloss: 0.103406\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4400]\tvalid_0's multi_logloss: 0.103227\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4450]\tvalid_0's multi_logloss: 0.102986\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4500]\tvalid_0's multi_logloss: 0.102788\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4550]\tvalid_0's multi_logloss: 0.102466\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4600]\tvalid_0's multi_logloss: 0.102287\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4650]\tvalid_0's multi_logloss: 0.102138\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4700]\tvalid_0's multi_logloss: 0.101951\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4750]\tvalid_0's multi_logloss: 0.101735\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4800]\tvalid_0's multi_logloss: 0.101611\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4850]\tvalid_0's multi_logloss: 0.101362\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4900]\tvalid_0's multi_logloss: 0.101221\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4950]\tvalid_0's multi_logloss: 0.101075\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[5000]\tvalid_0's multi_logloss: 0.10093\n",
      "Did not meet early stopping. Best iteration is:\n",
      "[4988]\tvalid_0's multi_logloss: 0.100912\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 20:03:19,270]\u001b[0m Trial 300 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:03:24,024]\u001b[0m Trial 301 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:03:29,003]\u001b[0m Trial 302 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:03:33,769]\u001b[0m Trial 303 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:03:39,428]\u001b[0m Trial 304 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:03:43,005]\u001b[0m Trial 305 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:03:49,279]\u001b[0m Trial 306 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:03:53,868]\u001b[0m Trial 307 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:03:59,420]\u001b[0m Trial 308 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:04:04,308]\u001b[0m Trial 309 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:04:41,514]\u001b[0m Trial 310 finished with value: 0.9448184199957927 and parameters: {'lambda_l1': 6.33710851183545e-08, 'lambda_l2': 0.00011828921524358206, 'num_leaves': 221, 'feature_fraction': 0.7490482980682972, 'bagging_fraction': 0.7149335066227929, 'bagging_freq': 1, 'min_child_samples': 75}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:04:47,751]\u001b[0m Trial 311 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:04:52,522]\u001b[0m Trial 312 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:05:03,194]\u001b[0m Trial 313 pruned. Trial was pruned at iteration 21.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:05:08,173]\u001b[0m Trial 314 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:05:13,677]\u001b[0m Trial 315 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:05:18,274]\u001b[0m Trial 316 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:05:24,488]\u001b[0m Trial 317 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:05:29,197]\u001b[0m Trial 318 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:05:33,932]\u001b[0m Trial 319 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:05:39,317]\u001b[0m Trial 320 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:05:43,765]\u001b[0m Trial 321 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:05:50,484]\u001b[0m Trial 322 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:05:54,773]\u001b[0m Trial 323 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:06:03,444]\u001b[0m Trial 324 pruned. Trial was pruned at iteration 18.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:06:32,442]\u001b[0m Trial 325 pruned. Trial was pruned at iteration 95.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:06:37,822]\u001b[0m Trial 326 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:06:42,014]\u001b[0m Trial 327 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:07:21,450]\u001b[0m Trial 328 finished with value: 0.9455738081122946 and parameters: {'lambda_l1': 3.217868638779184e-05, 'lambda_l2': 5.750590102813345e-05, 'num_leaves': 226, 'feature_fraction': 0.7826114441623828, 'bagging_fraction': 0.6679740332158755, 'bagging_freq': 1, 'min_child_samples': 82}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:07:27,179]\u001b[0m Trial 329 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:07:32,088]\u001b[0m Trial 330 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:07:37,170]\u001b[0m Trial 331 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:07:42,279]\u001b[0m Trial 332 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:07:46,023]\u001b[0m Trial 333 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:07:53,069]\u001b[0m Trial 334 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:07:57,424]\u001b[0m Trial 335 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:08:25,834]\u001b[0m Trial 336 pruned. Trial was pruned at iteration 78.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:08:30,146]\u001b[0m Trial 337 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:08:35,020]\u001b[0m Trial 338 pruned. Trial was pruned at iteration 12.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:08:41,479]\u001b[0m Trial 339 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:08:45,294]\u001b[0m Trial 340 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:08:51,071]\u001b[0m Trial 341 pruned. Trial was pruned at iteration 12.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:08:56,090]\u001b[0m Trial 342 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:09:01,151]\u001b[0m Trial 343 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:09:05,433]\u001b[0m Trial 344 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:09:42,571]\u001b[0m Trial 345 finished with value: 0.9442255837524621 and parameters: {'lambda_l1': 1.4659439917902066e-06, 'lambda_l2': 9.096938742062834e-06, 'num_leaves': 223, 'feature_fraction': 0.78931856475032, 'bagging_fraction': 0.687754997409419, 'bagging_freq': 1, 'min_child_samples': 92}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:09:47,319]\u001b[0m Trial 346 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:09:52,425]\u001b[0m Trial 347 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:09:57,385]\u001b[0m Trial 348 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:10:01,115]\u001b[0m Trial 349 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:10:07,462]\u001b[0m Trial 350 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:10:11,891]\u001b[0m Trial 351 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:10:17,813]\u001b[0m Trial 352 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:10:22,777]\u001b[0m Trial 353 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:10:28,182]\u001b[0m Trial 354 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:10:34,019]\u001b[0m Trial 355 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:10:38,990]\u001b[0m Trial 356 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:10:49,735]\u001b[0m Trial 357 pruned. Trial was pruned at iteration 31.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:10:55,530]\u001b[0m Trial 358 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:10:59,834]\u001b[0m Trial 359 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:11:04,538]\u001b[0m Trial 360 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:11:09,697]\u001b[0m Trial 361 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:11:47,302]\u001b[0m Trial 362 finished with value: 0.9451052762425657 and parameters: {'lambda_l1': 0.17345143397850926, 'lambda_l2': 1.530418515864943e-05, 'num_leaves': 227, 'feature_fraction': 0.7794541612455614, 'bagging_fraction': 0.8921910315517494, 'bagging_freq': 1, 'min_child_samples': 71}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:11:52,146]\u001b[0m Trial 363 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:11:56,135]\u001b[0m Trial 364 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:12:00,790]\u001b[0m Trial 365 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:12:05,348]\u001b[0m Trial 366 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:12:10,259]\u001b[0m Trial 367 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:12:14,597]\u001b[0m Trial 368 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:12:21,394]\u001b[0m Trial 369 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:12:25,610]\u001b[0m Trial 370 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:12:31,248]\u001b[0m Trial 371 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:12:35,702]\u001b[0m Trial 372 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:12:41,410]\u001b[0m Trial 373 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:12:49,192]\u001b[0m Trial 374 pruned. Trial was pruned at iteration 22.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:12:54,613]\u001b[0m Trial 375 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:12:59,016]\u001b[0m Trial 376 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 20:13:20,403]\u001b[0m Trial 377 pruned. Trial was pruned at iteration 65.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:13:25,274]\u001b[0m Trial 378 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:13:31,727]\u001b[0m Trial 379 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:13:36,235]\u001b[0m Trial 380 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:14:13,363]\u001b[0m Trial 381 finished with value: 0.9444646306247729 and parameters: {'lambda_l1': 0.00013491606868735477, 'lambda_l2': 2.4016424269924617e-06, 'num_leaves': 224, 'feature_fraction': 0.7645040960635633, 'bagging_fraction': 0.6776788045685024, 'bagging_freq': 1, 'min_child_samples': 91}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:14:21,686]\u001b[0m Trial 382 pruned. Trial was pruned at iteration 17.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:14:26,224]\u001b[0m Trial 383 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:14:30,257]\u001b[0m Trial 384 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:14:35,262]\u001b[0m Trial 385 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:14:40,837]\u001b[0m Trial 386 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:14:46,456]\u001b[0m Trial 387 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:15:23,408]\u001b[0m Trial 388 finished with value: 0.9461570824807328 and parameters: {'lambda_l1': 0.03582500455746271, 'lambda_l2': 1.361737608270606e-06, 'num_leaves': 236, 'feature_fraction': 0.7947192105026746, 'bagging_fraction': 0.7121032476222902, 'bagging_freq': 1, 'min_child_samples': 85}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:15:29,081]\u001b[0m Trial 389 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:15:35,152]\u001b[0m Trial 390 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:15:40,204]\u001b[0m Trial 391 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:15:45,684]\u001b[0m Trial 392 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:15:49,921]\u001b[0m Trial 393 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:15:54,703]\u001b[0m Trial 394 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:16:00,737]\u001b[0m Trial 395 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:16:05,612]\u001b[0m Trial 396 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:16:09,888]\u001b[0m Trial 397 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:16:14,330]\u001b[0m Trial 398 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:16:19,744]\u001b[0m Trial 399 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of finished trials: 400\n",
      "Best trial:\n",
      "  Value: 0.8423533686485246\n",
      "  Params: \n",
      "    lambda_l1: 6.031700696477528e-05\n",
      "    lambda_l2: 0.24394450720600092\n",
      "    num_leaves: 33\n",
      "    feature_fraction: 0.48861458318031864\n",
      "    bagging_fraction: 0.9555014696451426\n",
      "    bagging_freq: 1\n",
      "    min_child_samples: 79\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.027687 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 2260\n",
      "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 52\n",
      "[LightGBM] [Info] Start training from score -1.009528\n",
      "[LightGBM] [Info] Start training from score -0.717945\n",
      "[LightGBM] [Info] Start training from score -2.789114\n",
      "[LightGBM] [Info] Start training from score -5.348131\n",
      "[LightGBM] [Info] Start training from score -4.112163\n",
      "[LightGBM] [Info] Start training from score -3.508220\n",
      "[LightGBM] [Info] Start training from score -3.343871\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's multi_logloss: 0.44539\n",
      "[100]\tvalid_0's multi_logloss: 0.378913\n",
      "[150]\tvalid_0's multi_logloss: 0.340888\n",
      "[200]\tvalid_0's multi_logloss: 0.31566\n",
      "[250]\tvalid_0's multi_logloss: 0.29524\n",
      "[300]\tvalid_0's multi_logloss: 0.278682\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[350]\tvalid_0's multi_logloss: 0.263956\n",
      "[400]\tvalid_0's multi_logloss: 0.252259\n",
      "[450]\tvalid_0's multi_logloss: 0.241152\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[500]\tvalid_0's multi_logloss: 0.231783\n",
      "[550]\tvalid_0's multi_logloss: 0.223767\n",
      "[600]\tvalid_0's multi_logloss: 0.215283\n",
      "[650]\tvalid_0's multi_logloss: 0.207762\n",
      "[700]\tvalid_0's multi_logloss: 0.201304\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[750]\tvalid_0's multi_logloss: 0.19581\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[800]\tvalid_0's multi_logloss: 0.190409\n",
      "[850]\tvalid_0's multi_logloss: 0.185526\n",
      "[900]\tvalid_0's multi_logloss: 0.18142\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[950]\tvalid_0's multi_logloss: 0.177042\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1000]\tvalid_0's multi_logloss: 0.173195\n",
      "[1050]\tvalid_0's multi_logloss: 0.168927\n",
      "[1100]\tvalid_0's multi_logloss: 0.165216\n",
      "[1150]\tvalid_0's multi_logloss: 0.16232\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1200]\tvalid_0's multi_logloss: 0.159091\n",
      "[1250]\tvalid_0's multi_logloss: 0.156049\n",
      "[1300]\tvalid_0's multi_logloss: 0.153303\n",
      "[1350]\tvalid_0's multi_logloss: 0.150912\n",
      "[1400]\tvalid_0's multi_logloss: 0.148654\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1450]\tvalid_0's multi_logloss: 0.1463\n",
      "[1500]\tvalid_0's multi_logloss: 0.144477\n",
      "[1550]\tvalid_0's multi_logloss: 0.142652\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1600]\tvalid_0's multi_logloss: 0.140806\n",
      "[1650]\tvalid_0's multi_logloss: 0.138759\n",
      "[1700]\tvalid_0's multi_logloss: 0.137062\n",
      "[1750]\tvalid_0's multi_logloss: 0.13547\n",
      "[1800]\tvalid_0's multi_logloss: 0.134167\n",
      "[1850]\tvalid_0's multi_logloss: 0.132412\n",
      "[1900]\tvalid_0's multi_logloss: 0.13077\n",
      "[1950]\tvalid_0's multi_logloss: 0.129542\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2000]\tvalid_0's multi_logloss: 0.128147\n",
      "[2050]\tvalid_0's multi_logloss: 0.126764\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2100]\tvalid_0's multi_logloss: 0.125437\n",
      "[2150]\tvalid_0's multi_logloss: 0.124289\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2200]\tvalid_0's multi_logloss: 0.123232\n",
      "[2250]\tvalid_0's multi_logloss: 0.122371\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2300]\tvalid_0's multi_logloss: 0.121494\n",
      "[2350]\tvalid_0's multi_logloss: 0.120514\n",
      "[2400]\tvalid_0's multi_logloss: 0.119568\n",
      "[2450]\tvalid_0's multi_logloss: 0.118828\n",
      "[2500]\tvalid_0's multi_logloss: 0.117923\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2550]\tvalid_0's multi_logloss: 0.116858\n",
      "[2600]\tvalid_0's multi_logloss: 0.116208\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2650]\tvalid_0's multi_logloss: 0.115557\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2700]\tvalid_0's multi_logloss: 0.114849\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2750]\tvalid_0's multi_logloss: 0.11405\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2800]\tvalid_0's multi_logloss: 0.113411\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2850]\tvalid_0's multi_logloss: 0.112686\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2900]\tvalid_0's multi_logloss: 0.11198\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2950]\tvalid_0's multi_logloss: 0.111297\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3000]\tvalid_0's multi_logloss: 0.11087\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3050]\tvalid_0's multi_logloss: 0.110211\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3100]\tvalid_0's multi_logloss: 0.109625\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3150]\tvalid_0's multi_logloss: 0.109116\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3200]\tvalid_0's multi_logloss: 0.108663\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3250]\tvalid_0's multi_logloss: 0.108126\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3300]\tvalid_0's multi_logloss: 0.107707\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3350]\tvalid_0's multi_logloss: 0.107195\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3400]\tvalid_0's multi_logloss: 0.106732\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3450]\tvalid_0's multi_logloss: 0.106219\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3500]\tvalid_0's multi_logloss: 0.105908\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3550]\tvalid_0's multi_logloss: 0.105555\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3600]\tvalid_0's multi_logloss: 0.105145\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3650]\tvalid_0's multi_logloss: 0.104808\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3700]\tvalid_0's multi_logloss: 0.104423\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3750]\tvalid_0's multi_logloss: 0.104159\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3800]\tvalid_0's multi_logloss: 0.103877\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3850]\tvalid_0's multi_logloss: 0.103484\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3900]\tvalid_0's multi_logloss: 0.103104\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3950]\tvalid_0's multi_logloss: 0.102836\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4000]\tvalid_0's multi_logloss: 0.102578\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4050]\tvalid_0's multi_logloss: 0.102273\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4100]\tvalid_0's multi_logloss: 0.101997\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4150]\tvalid_0's multi_logloss: 0.101719\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4200]\tvalid_0's multi_logloss: 0.101443\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4250]\tvalid_0's multi_logloss: 0.101108\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4300]\tvalid_0's multi_logloss: 0.100871\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4350]\tvalid_0's multi_logloss: 0.10061\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4400]\tvalid_0's multi_logloss: 0.100397\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4450]\tvalid_0's multi_logloss: 0.100218\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4500]\tvalid_0's multi_logloss: 0.100004\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4550]\tvalid_0's multi_logloss: 0.0997785\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4600]\tvalid_0's multi_logloss: 0.0996502\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4650]\tvalid_0's multi_logloss: 0.0993899\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4700]\tvalid_0's multi_logloss: 0.0991771\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4750]\tvalid_0's multi_logloss: 0.0989407\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4800]\tvalid_0's multi_logloss: 0.0987919\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4850]\tvalid_0's multi_logloss: 0.0985558\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4900]\tvalid_0's multi_logloss: 0.0983888\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4950]\tvalid_0's multi_logloss: 0.098254\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[5000]\tvalid_0's multi_logloss: 0.0981239\n",
      "Did not meet early stopping. Best iteration is:\n",
      "[5000]\tvalid_0's multi_logloss: 0.0981239\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 20:31:41,953]\u001b[0m Trial 400 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:31:46,431]\u001b[0m Trial 401 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:31:50,256]\u001b[0m Trial 402 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:31:56,970]\u001b[0m Trial 403 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:32:03,528]\u001b[0m Trial 404 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:32:07,953]\u001b[0m Trial 405 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:32:13,149]\u001b[0m Trial 406 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:32:18,395]\u001b[0m Trial 407 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:32:23,606]\u001b[0m Trial 408 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:32:29,333]\u001b[0m Trial 409 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:32:33,493]\u001b[0m Trial 410 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:33:10,283]\u001b[0m Trial 411 finished with value: 0.9432215868887571 and parameters: {'lambda_l1': 1.0106430243212002e-07, 'lambda_l2': 1.7338808138171514e-05, 'num_leaves': 211, 'feature_fraction': 0.8525200092071654, 'bagging_fraction': 0.71247365704883, 'bagging_freq': 1, 'min_child_samples': 77}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:33:15,221]\u001b[0m Trial 412 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:33:20,036]\u001b[0m Trial 413 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:33:32,811]\u001b[0m Trial 414 pruned. Trial was pruned at iteration 35.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:33:36,360]\u001b[0m Trial 415 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:34:13,257]\u001b[0m Trial 416 finished with value: 0.944780172496223 and parameters: {'lambda_l1': 6.61840029683868e-08, 'lambda_l2': 6.189598777012943e-05, 'num_leaves': 229, 'feature_fraction': 0.7543055044592567, 'bagging_fraction': 0.942324205989867, 'bagging_freq': 1, 'min_child_samples': 88}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:34:18,909]\u001b[0m Trial 417 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:34:23,665]\u001b[0m Trial 418 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:34:29,134]\u001b[0m Trial 419 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:34:33,970]\u001b[0m Trial 420 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:34:39,344]\u001b[0m Trial 421 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:34:44,235]\u001b[0m Trial 422 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:34:47,811]\u001b[0m Trial 423 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:34:53,765]\u001b[0m Trial 424 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:34:59,389]\u001b[0m Trial 425 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:35:04,937]\u001b[0m Trial 426 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:35:10,624]\u001b[0m Trial 427 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:35:16,512]\u001b[0m Trial 428 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:35:22,165]\u001b[0m Trial 429 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:35:27,930]\u001b[0m Trial 430 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:35:33,197]\u001b[0m Trial 431 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:35:38,098]\u001b[0m Trial 432 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:35:43,682]\u001b[0m Trial 433 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:35:48,031]\u001b[0m Trial 434 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:35:53,726]\u001b[0m Trial 435 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:35:58,420]\u001b[0m Trial 436 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:36:03,775]\u001b[0m Trial 437 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:36:09,687]\u001b[0m Trial 438 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:36:14,370]\u001b[0m Trial 439 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:36:19,980]\u001b[0m Trial 440 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:36:23,347]\u001b[0m Trial 441 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:36:30,224]\u001b[0m Trial 442 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:36:34,705]\u001b[0m Trial 443 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:36:39,922]\u001b[0m Trial 444 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:36:45,883]\u001b[0m Trial 445 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:36:49,825]\u001b[0m Trial 446 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:36:56,073]\u001b[0m Trial 447 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:37:00,642]\u001b[0m Trial 448 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:37:05,855]\u001b[0m Trial 449 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:37:10,406]\u001b[0m Trial 450 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:37:15,960]\u001b[0m Trial 451 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:37:22,067]\u001b[0m Trial 452 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:37:26,480]\u001b[0m Trial 453 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:37:31,630]\u001b[0m Trial 454 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:37:36,056]\u001b[0m Trial 455 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:37:40,816]\u001b[0m Trial 456 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:37:46,124]\u001b[0m Trial 457 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:37:50,577]\u001b[0m Trial 458 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:37:58,422]\u001b[0m Trial 459 pruned. Trial was pruned at iteration 14.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:38:02,782]\u001b[0m Trial 460 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:38:09,254]\u001b[0m Trial 461 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:38:13,847]\u001b[0m Trial 462 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:38:19,221]\u001b[0m Trial 463 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:38:23,415]\u001b[0m Trial 464 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:38:28,284]\u001b[0m Trial 465 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:38:33,838]\u001b[0m Trial 466 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:38:38,296]\u001b[0m Trial 467 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:38:43,969]\u001b[0m Trial 468 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:38:48,921]\u001b[0m Trial 469 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:38:54,443]\u001b[0m Trial 470 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:38:59,568]\u001b[0m Trial 471 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:39:04,433]\u001b[0m Trial 472 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:39:10,663]\u001b[0m Trial 473 pruned. Trial was pruned at iteration 11.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:39:14,889]\u001b[0m Trial 474 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:39:20,884]\u001b[0m Trial 475 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:39:25,563]\u001b[0m Trial 476 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:39:30,567]\u001b[0m Trial 477 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:39:36,476]\u001b[0m Trial 478 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:39:39,231]\u001b[0m Trial 479 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:39:44,485]\u001b[0m Trial 480 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:39:49,432]\u001b[0m Trial 481 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:39:54,124]\u001b[0m Trial 482 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-09-07 20:39:58,219]\u001b[0m Trial 483 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:40:02,806]\u001b[0m Trial 484 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:40:09,290]\u001b[0m Trial 485 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:40:49,570]\u001b[0m Trial 486 finished with value: 0.9436040618844543 and parameters: {'lambda_l1': 0.00020397245265135663, 'lambda_l2': 9.093796518313558e-06, 'num_leaves': 214, 'feature_fraction': 0.8663304164529336, 'bagging_fraction': 0.6476610679220174, 'bagging_freq': 4, 'min_child_samples': 87}. Best is trial 3 with value: 0.8423533686485246.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:40:54,931]\u001b[0m Trial 487 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:41:00,439]\u001b[0m Trial 488 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:41:05,860]\u001b[0m Trial 489 pruned. Trial was pruned at iteration 13.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:41:10,637]\u001b[0m Trial 490 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:41:15,141]\u001b[0m Trial 491 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:41:19,440]\u001b[0m Trial 492 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:41:23,700]\u001b[0m Trial 493 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:41:28,019]\u001b[0m Trial 494 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:41:33,694]\u001b[0m Trial 495 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:41:51,434]\u001b[0m Trial 496 pruned. Trial was pruned at iteration 56.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:41:57,024]\u001b[0m Trial 497 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:42:02,000]\u001b[0m Trial 498 pruned. Trial was pruned at iteration 10.\u001b[0m\n",
      "\u001b[32m[I 2021-09-07 20:42:07,346]\u001b[0m Trial 499 pruned. Trial was pruned at iteration 10.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of finished trials: 500\n",
      "Best trial:\n",
      "  Value: 0.8423533686485246\n",
      "  Params: \n",
      "    lambda_l1: 6.031700696477528e-05\n",
      "    lambda_l2: 0.24394450720600092\n",
      "    num_leaves: 33\n",
      "    feature_fraction: 0.48861458318031864\n",
      "    bagging_fraction: 0.9555014696451426\n",
      "    bagging_freq: 1\n",
      "    min_child_samples: 79\n",
      "[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.025451 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 2268\n",
      "[LightGBM] [Info] Number of data points in the train set: 418328, number of used features: 53\n",
      "[LightGBM] [Info] Start training from score -1.009528\n",
      "[LightGBM] [Info] Start training from score -0.717945\n",
      "[LightGBM] [Info] Start training from score -2.789114\n",
      "[LightGBM] [Info] Start training from score -5.348131\n",
      "[LightGBM] [Info] Start training from score -4.112163\n",
      "[LightGBM] [Info] Start training from score -3.508220\n",
      "[LightGBM] [Info] Start training from score -3.343871\n",
      "Training until validation scores don't improve for 20 rounds\n",
      "[50]\tvalid_0's multi_logloss: 0.44195\n",
      "[100]\tvalid_0's multi_logloss: 0.375935\n",
      "[150]\tvalid_0's multi_logloss: 0.338811\n",
      "[200]\tvalid_0's multi_logloss: 0.313199\n",
      "[250]\tvalid_0's multi_logloss: 0.294071\n",
      "[300]\tvalid_0's multi_logloss: 0.277107\n",
      "[350]\tvalid_0's multi_logloss: 0.264836\n",
      "[400]\tvalid_0's multi_logloss: 0.252349\n",
      "[450]\tvalid_0's multi_logloss: 0.241412\n",
      "[500]\tvalid_0's multi_logloss: 0.231972\n",
      "[550]\tvalid_0's multi_logloss: 0.22405\n",
      "[600]\tvalid_0's multi_logloss: 0.217065\n",
      "[650]\tvalid_0's multi_logloss: 0.210183\n",
      "[700]\tvalid_0's multi_logloss: 0.204259\n",
      "[750]\tvalid_0's multi_logloss: 0.198327\n",
      "[800]\tvalid_0's multi_logloss: 0.192336\n",
      "[850]\tvalid_0's multi_logloss: 0.187595\n",
      "[900]\tvalid_0's multi_logloss: 0.183324\n",
      "[950]\tvalid_0's multi_logloss: 0.178581\n",
      "[1000]\tvalid_0's multi_logloss: 0.174766\n",
      "[1050]\tvalid_0's multi_logloss: 0.171164\n",
      "[1100]\tvalid_0's multi_logloss: 0.167795\n",
      "[1150]\tvalid_0's multi_logloss: 0.164106\n",
      "[1200]\tvalid_0's multi_logloss: 0.160688\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1250]\tvalid_0's multi_logloss: 0.157639\n",
      "[1300]\tvalid_0's multi_logloss: 0.155048\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1350]\tvalid_0's multi_logloss: 0.152564\n",
      "[1400]\tvalid_0's multi_logloss: 0.14986\n",
      "[1450]\tvalid_0's multi_logloss: 0.147446\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1500]\tvalid_0's multi_logloss: 0.145395\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1550]\tvalid_0's multi_logloss: 0.143255\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1600]\tvalid_0's multi_logloss: 0.141166\n",
      "[1650]\tvalid_0's multi_logloss: 0.139344\n",
      "[1700]\tvalid_0's multi_logloss: 0.137731\n",
      "[1750]\tvalid_0's multi_logloss: 0.136312\n",
      "[1800]\tvalid_0's multi_logloss: 0.134798\n",
      "[1850]\tvalid_0's multi_logloss: 0.133435\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[1900]\tvalid_0's multi_logloss: 0.131893\n",
      "[1950]\tvalid_0's multi_logloss: 0.130715\n",
      "[2000]\tvalid_0's multi_logloss: 0.129298\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2050]\tvalid_0's multi_logloss: 0.127718\n",
      "[2100]\tvalid_0's multi_logloss: 0.126323\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2150]\tvalid_0's multi_logloss: 0.125164\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2200]\tvalid_0's multi_logloss: 0.124135\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2250]\tvalid_0's multi_logloss: 0.123227\n",
      "[2300]\tvalid_0's multi_logloss: 0.122213\n",
      "[2350]\tvalid_0's multi_logloss: 0.121182\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2400]\tvalid_0's multi_logloss: 0.120412\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2450]\tvalid_0's multi_logloss: 0.119649\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2500]\tvalid_0's multi_logloss: 0.118669\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2550]\tvalid_0's multi_logloss: 0.11796\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2600]\tvalid_0's multi_logloss: 0.11723\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2650]\tvalid_0's multi_logloss: 0.116418\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2700]\tvalid_0's multi_logloss: 0.11571\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2750]\tvalid_0's multi_logloss: 0.114968\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2800]\tvalid_0's multi_logloss: 0.114305\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2850]\tvalid_0's multi_logloss: 0.113767\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[2900]\tvalid_0's multi_logloss: 0.113049\n",
      "[2950]\tvalid_0's multi_logloss: 0.112492\n",
      "[3000]\tvalid_0's multi_logloss: 0.111726\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3050]\tvalid_0's multi_logloss: 0.111202\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3100]\tvalid_0's multi_logloss: 0.110604\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3150]\tvalid_0's multi_logloss: 0.110046\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3200]\tvalid_0's multi_logloss: 0.109533\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3250]\tvalid_0's multi_logloss: 0.109078\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3300]\tvalid_0's multi_logloss: 0.108525\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3350]\tvalid_0's multi_logloss: 0.108166\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3400]\tvalid_0's multi_logloss: 0.107835\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3450]\tvalid_0's multi_logloss: 0.107511\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3500]\tvalid_0's multi_logloss: 0.107145\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3550]\tvalid_0's multi_logloss: 0.106771\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3600]\tvalid_0's multi_logloss: 0.106495\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3650]\tvalid_0's multi_logloss: 0.106142\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3700]\tvalid_0's multi_logloss: 0.105776\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3750]\tvalid_0's multi_logloss: 0.105335\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3800]\tvalid_0's multi_logloss: 0.104977\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3850]\tvalid_0's multi_logloss: 0.104609\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3900]\tvalid_0's multi_logloss: 0.104318\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[3950]\tvalid_0's multi_logloss: 0.104044\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4000]\tvalid_0's multi_logloss: 0.10378\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4050]\tvalid_0's multi_logloss: 0.103526\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4100]\tvalid_0's multi_logloss: 0.103203\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4150]\tvalid_0's multi_logloss: 0.10292\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4200]\tvalid_0's multi_logloss: 0.102592\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4250]\tvalid_0's multi_logloss: 0.102358\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4300]\tvalid_0's multi_logloss: 0.102136\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4350]\tvalid_0's multi_logloss: 0.101919\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4400]\tvalid_0's multi_logloss: 0.101602\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4450]\tvalid_0's multi_logloss: 0.101265\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4500]\tvalid_0's multi_logloss: 0.10099\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4550]\tvalid_0's multi_logloss: 0.100813\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4600]\tvalid_0's multi_logloss: 0.100597\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4650]\tvalid_0's multi_logloss: 0.100381\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4700]\tvalid_0's multi_logloss: 0.100161\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4750]\tvalid_0's multi_logloss: 0.100034\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4800]\tvalid_0's multi_logloss: 0.0998763\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4850]\tvalid_0's multi_logloss: 0.0996593\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4900]\tvalid_0's multi_logloss: 0.099492\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[4950]\tvalid_0's multi_logloss: 0.0993946\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[5000]\tvalid_0's multi_logloss: 0.099238\n",
      "Did not meet early stopping. Best iteration is:\n",
      "[5000]\tvalid_0's multi_logloss: 0.099238\n",
      "Wall time: 2h 35min 57s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "if __name__ == \"__main__\":\n",
    "    test_predict = 0\n",
    "    study = optuna.create_study(\n",
    "        pruner=optuna.pruners.MedianPruner(n_warmup_steps=10), direction=\"minimize\"  # 指定是越小越好\n",
    "    )\n",
    "    for i, (trn_idx, val_idx) in enumerate(skf.split(train_x, train_y)):\n",
    "        study.optimize(lambda trial: objective(trial, train_x.iloc[trn_idx], train_y.iloc[trn_idx], \n",
    "                                               train_x.iloc[val_idx], train_y.iloc[val_idx]), n_trials=100)\n",
    "\n",
    "        print(\"Number of finished trials: {}\".format(len(study.trials)))\n",
    "\n",
    "        print(\"Best trial:\")\n",
    "        trial = study.best_trial\n",
    "\n",
    "        print(\"  Value: {}\".format(trial.value))\n",
    "\n",
    "        print(\"  Params: \")\n",
    "        for key, value in trial.params.items():\n",
    "            print(\"    {}: {}\".format(key, value))\n",
    "            \n",
    "        params = {\"boosting_type\": \"gbdt\",\n",
    "                  \"objective\": \"multiclass\",\n",
    "                  \"num_class\": 7,\n",
    "                  \"metric\": \"multi_logloss\"}\n",
    "        for key, value in trial.params.items():\n",
    "            params[key]=value\n",
    "            \n",
    "        dtrain = lgb.Dataset(train_x.iloc[trn_idx], label=train_y.iloc[trn_idx])\n",
    "        dvalid = lgb.Dataset(train_x.iloc[val_idx], label=train_y.iloc[val_idx])\n",
    "        model = lgb.train(params=params, train_set=dtrain,valid_sets=[dvalid],\n",
    "                          verbose_eval=50,\n",
    "                          early_stopping_rounds=20,\n",
    "                          num_boost_round=5000)\n",
    "        test_predict += model.predict(test_x)\n",
    "    predict = test_predict/5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "d0cad2c6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "使用optuna的多分类分数： 0.968056177067915\n"
     ]
    }
   ],
   "source": [
    "# 评估指标acc，越大越好\n",
    "print('使用optuna的多分类分数：', accuracy_score(np.argmax(predict, axis=1), test_y))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63f90cf8",
   "metadata": {},
   "source": [
    "### 分类任务的结论\n",
    "不使用optuna的ACC分数是0.8631716636260369，使用的ACC分数是0.968056177067915，提升了0.1048845134418781。\n",
    "\n",
    "结合Stacking.ipynb里的单模随机森林最好分数0.9535087055000585甚至Stacking后的融合分数0.9589887988544127进行对比，也都是上涨的。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
